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Accompanying Statement from the Independent Advisory Panel 


To: Chairman John Kline, Committee on Education and the Workforce, U.S. House of 
Representatives 
Chairman Tom Harkin, Committee on Health, Education, Labor, and Pensions, U.S. 
Senate 


From: Independent Advisory Panel of the National Assessment of Career and Technical 
Education 


Re: Interim Report of the National Assessment 


The Carl D. Perkins Career and Technical Education Improvement Act of 2006 (Perkins IV) 
called on the secretary of education to appoint an Independent Advisory Panel (IAP) to provide 
guidance on the topics and methodology of the National Assessment of Career and Technical 
Education (NACTE), and to provide an independent report on the findings of the assessment. 
The IAP, which includes experts in career and technical education as well as experts in research 
techniques and methodology, has met four times in Washington to provide guidance and 
feedback on the NACTE studies. We have also reviewed the NACTE Interim Report and the 
studies on which it is based. 


We are writing at this time to draw your attention to key issues that emerge from the Interim 
Report, which we trust you will read with great interest. 


Changing Landscape of Career and Technical Education 


Just as 21st-century workforce needs are changing what it means to be prepared in career and 
technical education (CTE), the population of students engaged in CTE 1s also shifting. Whereas 
three decades ago, secondary occupational courses were primarily the province of students 
without a strong academic orientation, the most recent data (from the high school class of 2004) 
reveal that students from across the academic spectrum are enrolled in both academic and 
occupational courses. Students participating in occupational courses are taking more academic 
credits than ever before, and low-income students make up a smaller share of course enrollments 
than in the past. Overall participation in CTE courses remained steady, but the proportion of 
students concentrating in CTE—those enrolled in three or more courses in the same field—has 
declined since the 1980s. These patterns are consistent with a view that secondary CTE has 
become less of a “track” for low-achieving, low-income students and more of a “field” in which 
students participate from a wide variety of backgrounds and interests. 


Meanwhile, the new realities of the workplace such as the increasing use of technology and the 
growth of the health care sector have occurred alongside an expansion of occupational 
preparation at the postsecondary level. These trends have been accompanied by a desire to create 
pathways of occupational studies that extend from the secondary to the postsecondary level. 
More students who invest in CTE at the secondary level (those who take three or more credits in 
any CTE field) are continuing to the postsecondary level, and the gap in postsecondary 
attendance between CTE investors and nonparticipants in secondary CTE has declined. 
However, data on whether students are actually following specific CTE pathways or “Programs 
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of Study” are not available, nor are they required by CTE accountability systems. Among 
students who do not attend college, moreover, national data show that CTE investors are more 
likely to find skilled jobs than are students who earn few or no occupational credits, an 
advantage for CTE students that has increased since the 1980s. 


Evaluation Framework and Evidence 


The NACTE has devised a framework for evaluation that is appropriate given its charge and 
resources. However, three main challenges have necessarily limited the scope of the evaluation 


* Timing: Because the 2006 act is just now being fully implemented, the NACTE will 
evaluate mainly the quality of implementation. Evaluation of impact will also be part of 
the NACTE, but impact estimates will mainly reflect conditions of CTE that were in 
place at the start of the 2006 act, not those that unfolded subsequently. 


¢ Longitudinal data: Until now, few states have had the longitudinal data necessary to 
permit evaluation of more than short-term outcomes; Florida is the noteworthy exception. 
As states move to comply with the education provisions of the American Recovery and 
Reinvestment Act, one may expect that longitudinal data will become more widely 
available. 


* Evidence of quality: Despite important new accountability provisions in the 2006 act, 
available data focus mainly on who takes which CTE programs and courses, and little 
evidence is available to assess the quality of CTE instruction. Case studies that will be 
discussed in the NACTE final report will begin to address this gap, but such information 
is not regularly gathered. 


Ensuring that students complete high school “college- and career-ready” has been identified as 
an important national policy goal. The IAP wishes to emphasize the potential for CTE to 
contribute meaningfully toward achieving this aim. Broader education reforms such improving 
teacher quality, turning around low-performing schools, setting high standards, and improving 
data systems should also be considered for CTE. Lessons may also be drawn from CTE that 
apply to education reform efforts more generally. Including CTE in broader conversations about 
education reform would ensure CTE is part of an overall improvement strategy and not isolated 
from other, complementary efforts to improve educational outcomes. 


Looking Toward the Future 


CTE has the potential to play a vital role in strengthening America’s competitiveness in the 
global economy. Because it spans the secondary and postsecondary levels, CTE can not only 
help students become “college- and career-ready,” it can encourage them to pursue 
postsecondary education or training, especially in high-skill, high-wage, and high-demand 
occupations. 
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Highlights 
Background 


Under the Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV), Congress 
mandated a National Assessment of Career and Technical Education (NACTE) to evaluate the 
implementation of the law and its outcomes. This interim report is the first of two reports. The 
purpose of this NACTE interim report is two-fold: 


¢ To describe the overall research approach to meeting the congressional mandate. 


* To present interim findings to date from the national assessment focused on participation, 
outcomes and international comparisons of secondary school career and technical 
education (CTE). 


The final NACTE report will provide a more complete and integrated perspective on the progress 
made in implementing Perkins IV. 


The findings in this interim report focus primarily on the role of secondary CTE in the 
preparation and transition of youths for college and careers. The report notes that a special 
challenge facing CTE in meeting these twin goals of academic and career preparation is the role 
of CTE for the approximately half the students in each secondary class who do not directly enter 
college upon leaving high school. Many of these students were having difficulty finding 
employment before the financial crisis and now employment is even more difficult. Based on 
Bureau of Labor Statistics data, a majority of recent graduates (54.3 percent) were jobless in 
2011 as were nearly two-thirds (65.6 percent) of recent high school dropouts. 


Findings 


1. Participation in secondary CTE changed significantly over the 22-year period between 1982 
and 2004. Although most students continue to take at least one CTE course, the focus of this 
report is on participation trends of students who invest in CTE (take three or more CTE courses). 


¢ The percentage of CTE investors among high school graduates initially declined from 
46 percent to 35 percent between 1982 and 1992 but stabilized and even increased 
slightly from 35 percent to 38 percent between 1992 and 2004. 


¢ CTE investors have substantially closed the academic gap with non-investors in taking 
core academic courses (“new basics’’) due in part by increasing academic course 
requirements as a condition for graduation in most states. By 2004, about 58 percent of 
CTE investors had completed the new basics, compared with about 61 percent of other 
students. Additionally, a small set of new courses that generally enrolled higher 
achieving students (e.g, in computer science and education) became classified as CTE. 


* In 1982, 65 percent of investors concentrated in one occupational area. By 2004, the 
situation reversed when a majority (55 percent) of CTE investors took CTE in more than 
one occupational area (exploring) rather than in a single occupational area 
(concentrating). Because states generally report performance data under Perkins IV for 
only CTE concentrators, which most states define as secondary students who have taken 
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at least three courses in a single CTE program area, Perkins IV accountability data 
reflects the performance of somewhat less than half of students who invest in CTE. 


2. The NACTE presented new evidence of secondary CTE outcomes from two studies, with 
mixed outcome findings. An analysis of the nationally-representative NCES Education 
Longitudinal Study (ELS:2002) failed to find any relationship between CTE coursetaking and 
student achievement on a standardized math assessment or school completion. However, one 
study in a large urban school district (The School District of Philadelphia) found a positive 
relationship between CTE school attendance in oversubscribed schools and course and school 
completion but no relationship between CTE school attendance and achievement. 


To further advance understanding of the potential effects of secondary CTE, evaluations would 
benefit from going beyond the approach of current studies that examine the outcomes of CTE 
without taking into account program variation or implementation quality. A more useful 
approach would be to focus on rigorously assessing the potential for CTE benefits from a defined 
and well-implemented set of programs. 


3. The Organization for Economic Cooperation and Development (OECD) data show that 
secondary students in competitor nations devote a higher share of their course work to secondary 
vocational education programs than the United States, with the exception of Canada. Using 
OECD standards of vocational enrollment, in most European countries examined, almost half or 
more of secondary school graduates enroll in a vocational-oriented program (in which they earn 
25 percent or more of their total credits). Even in Korea and Japan, with their strong academic 
traditions, a quarter of secondary graduates concentrate in vocational education by OECD 
standards. By contrast, since the early 1980s the percentage of U.S. secondary students meeting 
OECD standards for vocational education enrollment declined from about 18 percent to 

6 percent. 


Implications 


Evidence from the interim report suggests that secondary CTE is shifting away from its historical 
dual-track approach and that successive cohorts of secondary CTE students are closing the gap 
with non-CTE students with respect to the number of core academic credits earned and college- 
going rates. 


The interim report also raises issues of the role of secondary CTE in preparing for work the 
nearly half the students who do not directly go on to postsecondary education. With about half of 
all secondary students not going directly to college, and many having difficulties finding 
employment, it is important that information on the effectiveness of potentially exemplary CTE 
be investigated. 
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Executive Summary 


The Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) reinforces a 
longstanding federal commitment to supporting career and technical education (CTE). Although 
federal contributions account for only a fraction of total state and local spending on CTE, federal 
policy has had, and continues to have, a large influence on state and local programs and policies. 
Over time, as national attention has turned to globalization and its implications for a more highly 
skilled workforce, so too has Perkins IV’s emphasis shifted, with current legislation aimed at 
raising the academic and technical rigor of secondary and postsecondary CTE instruction to 
prepare students for entry into high-skill, high-wage, or high-demand occupations. 


Perkins IV attempts to strengthen CTE and improve its quality primarily by increasing the 
emphasis on accountability and introducing programs of study (POS) that, among other things, 
integrate and align secondary and postsecondary levels across challenging and rigorous academic 
and technical content. 


Under Perkins IV, Congress mandated a National Assessment of Career and Technical Education 
(NACTE) to evaluate the implementation of the law and its outcomes. This interim report is the 
first of two reports. The purpose of this NACTE interim report is two-fold: 


* To describe the overall research approach to meeting the congressional mandate. 


¢ To present interim findings to date from the national assessment focused on participation, 
outcomes and international comparisons of secondary school CTE. 


The final NACTE report will provide a more complete and integrated perspective on the progress 
made in implementing Perkins IV. The final report will also include more detailed information 
about the outcomes of CTE participation, drawing upon several NACTE studies. The complete 
agenda of NACTE studies is listed in Exhibit ES-4 at the end of this executive summary. 


ES-1 Congressional Mandate for the NACTE 


Perkins IV directs the secretary of education to “provide for the conduct of an independent 
evaluation and assessment of career and technical education programs under the Act, including 
the implementation of the [Act].” Following the enactment of the new law, the secretary of 
education directed the Office of Planning, Evaluation and Policy Development (OPEPD) to 
conduct the NACTE. 


NACTE’s evaluation agenda is driven by the requirements in the congressional mandate to 
examine the implementation of provisions in Perkins IV and the outcomes of CTE. The 
congressionally mandated issues and the set of NACTE studies to address the mandate are, as 
follows: 


1. How have secondary and postsecondary student enrollments in CTE programs 
changed? Examples of studies providing data for the NACTE include secondary data 
analysis of the National Center for Education Statistics (NCES) longitudinal data from 
the Education Longitudinal Study of 2002 (ELS:2002), the National Education 
Longitudinal Study (NELS:88), the High School and Beyond (HS&B) study, the National 
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Assessment of Educational Progress (NAEP) High School Transcript Studies, and the 
Integrated Postsecondary Education Data System (IPEDS). 


2. Do students who participate in CTE realize any educational or workforce benefits? 
NACTE studies include secondary data analysis of ELS:2002 and secondary data 
analysis of natural experiments and quasi-experiments using local administrative data in 
Philadelphia and San Diego and state administrative data from Florida. 


3. How are Perkins IV funds distributed and used? NACTE studies include analysis of 
subgrantee fiscal data; state, district, and postsecondary institution surveys; and case 
studies of selected states, districts, schools, and postsecondary institutions. 


4. To what extent are Perkins IV accountability data valid, reliable, and comparable, 
and to what extent are decision-makers using these data? NACTE studies include 
state, district, and postsecondary institution surveys; case studies of selected states, 
districts, schools, and postsecondary institutions; and analysis of Perkins IV performance 
data systems. 


5. Are grantees meeting the requirement to develop and implement programs of study 
(POS) that integrate academic and technical content? NACTE studies include state, 
district, and postsecondary institution surveys; expert panel review of state POS; and case 
studies of selected states, districts, schools, and postsecondary institutions. 


Thus, NACTE is sponsoring a set of interrelated but distinct studies to address the congressional 
requirements and associated research questions. The overall agenda calls for diverse data 
collection and analytic methods: qualitative case studies, national surveys of states and local 
entities, use of national longitudinal and cross-sectional databases, use of state and local 
administrative databases, an expert panel, and both a natural experiment in choice of CTE 
schools and econometric estimation of national and local program effects. A detailed listing of 
NACTE study activities in relation to individual provisions in the legislative mandate is 
presented at the end of this executive summary. 


Results from the NACTE are to be reported in interim and final reports. This interim report will 
address parts of the first and second research questions in the preceding list: the participation of 
students in secondary CTE programs and a first look at the academic outcomes of CTE from two 
of NACTE’s quantitative studies. The final report will address the other topics. 


Independent Advisory Panel 


Congress further mandated that an Independent Advisory Panel (IAP) be formed for NACTE. 
The purposes of the IAP are to— 


¢ Provide guidance to the secretary of education on the implementation of the NACTE, 
including the issues to be addressed and the methodology of its research studies to ensure 
that they meet the highest standards of quality. 


¢ Prepare an independent analysis of the findings and recommendations resulting from the 
assessment. 
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The U.S. Department of Education has assembled an IAP whose members have a wide range of 
expertise (see p. 1x for a list of the members). 


Interim Report: A Special Focus on Secondary CTE 


The findings in this interim report focus primarily on the role of secondary CTE in the 
preparation and transition of youths for college and careers. While postsecondary CTE plays a 
critical role in workforce development and specialized skills training, the final report will 
examine this area more fully when the NACTE studies on postsecondary education have been 
completed. 


For a large part of the 20th century, CTE (formerly vocational education) was a mainstay in the 
career readiness preparation of the majority of secondary students who entered the labor force 
directly from high school. Those going on to college were schooled in the academic track and 
exposed to a more rigorous set of college preparation courses. Vocational majors in secondary 
school were placed on a track that was more occupationally focused. As greater percentages of 
high school graduates enrolled in college directly from high school, this dual track system 
resulted in increasingly negative perceptions of vocational education as programs where students 
were not adequately prepared for further education at the postsecondary level. Moreover, 
employers raised their expectations that new job entrants would bring strong basic academic 
skills, including high school graduates who were firmly grounded in reading, writing and 
mathematics skills (Levy and Murnane, 2004). 


Congress responded over the years by moving to modernize the federal vocational education law 
to meet the newer and more rigorous education and workplace requirements and eliminate the 
sharp separation between the academic and CTE course work under the dual track approach. The 
strengthening of the integration of academic and technical tracks continued in Perkins IV. One 
key feature is the introduction of POS, which as noted, establish in different occupational areas 
formal course sequences connecting high school and college work. POS combine academic and 
career-related courses and lead to credentials or certificates of attainment. Perkins IV also 
requires states and subgrantees to report on performance standards around core academic, 
technical, and graduation and placement indicators. 


This strategy of integration between academic and technical content also seeks to prepare 
secondary CTE students for postsecondary education and careers. The movement over the years 
has been to strengthen secondary CTE students’ access to a sound academic program while 
maintaining a strong technical program. This should permit students access without wasteful 
remediation at the postsecondary level while also building a foundation toward “high-skill, high- 
wage, or high-demand occupations in current or emerging professions.” 


A special challenge facing CTE in meeting these twin goals of academic and career preparation 
is the role of CTE for the approximately half the students in each secondary class who do not 
directly enter college upon leaving high school. These include the approximately one-third of the 


' Perkins IV, S 250-2. 
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high school graduating class who currently do not directly go on to postsecondary education as 
well as students who drop out of high school before graduation.’ 


To gain a measure of the challenge in preparing students for the workforce, the employment of 
recent high school graduates and dropouts declined sharply between 2007 and 2011. Since the 
financial crisis, the labor force participation rate of high school graduates is down, the 
unemployment rate is up, and a majority of graduates not enrolled in college (54.3 percent) are 
jobless compared to 38.6 percent in 2007.7 


The picture for high school dropouts is starker. Between 2007 and 2011, the unemployment rate 
for those looking for work has risen from 26.9 percent to 38.4 percent of high school dropouts. 
The jobless rate, which includes those who are not actively seeking work, went from 58.9 
percent in 2007 to 65.6 percent in 2011. 


To better understand the potential role of secondary CTE, this interim report will present 
information on the participation of secondary students in CTE using a variety of participation 
measures and examine the coursetaking and academic outcomes of secondary CTE students. The 
interim report will also compare CTE preparation in the U.S. to other industrialized nations. 


Organization of Interim Report 


This interim report provides baseline information and lays the groundwork for a comprehensive 
final report. It consists of six chapters: 


¢ Chapter 1 provides an overview of the NACTE research agenda. 


* Chapter 2 provides an introduction to the legislative provisions of Perkins IV and issues 
they raise for NACTE’s evaluation of the implementation and outcomes of the law. 


* Chapter 3 examines trends in student participation in CTE before Perkins IV was enacted 
using data from longitudinal studies of high school graduates conducted by the U.S. 
Department of Education’s NCES. 


* Chapter 4 summarizes prior studies of CTE outcomes and presents preliminary findings 
from two NACTE studies (a national study and a large city study) of secondary CTE 
outcomes associated with earlier versions of the Perkins Act. 


* Chapter 5 presents international benchmark data on pre—Perkins IV CTE participation 
and experiences comparing U.S. secondary students with their peers in other developed 
countries and drawing upon a report from the Organization for Economic Co-operation 
and Development (OECD, 2008). 


* Chapter 6 draws implications for the final report. 


* Of the 3.1 million youths age 16 to 24 who graduated from high school in January through October 2011, 
2.1 million went directly on to college and one million (31.8 percent) were not enrolled in college in October 2011. 


> Bureau of Labor Statistics. College enrollment and work activity of high school graduates news releases. Analysis 
of Current Population Survey data, 2007 and 2011. http://www.bls.gov/schedule/archives/all_nr.htm#HSGEC. 
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The Chapter 1 overview of NACTE has been discussed above, so that the remainder of this 
executive summary summarizes Chapters 2-6. 


ES-2 Chapter 2: Framework for Evaluating the Implementation of Perkins IV 
Provisions on Accountability and Programs of Study 


Chapter 2 provides an analytic perspective on the provisions in Perkins IV aimed at improving 
the quality of CTE. The framework employed is called a logic model because it looks at the 
implicit logic in the program’s structures and requirements found in Perkins IV. By identifying 
the key elements in the law, the sequences and causal assumptions about the way they operate 
and interact, and the incentives and requirements for required actions, the logic model and theory 
of action provide direction for what NACTE should be examining in assessing whether 

Perkins IV is achieving its intended effects. 


Perkins IV employs three major policy avenues for improving CTE services: (1) POS defining 
the expectations of a comprehensive sequence of CTE instruction; (2) accountability provisions 
holding state agencies and local subgrantees responsible for their performance; and (3) spending 
directives detailing the purposes and required uses of federal funds. These three legislative 
themes provide a range of policy levers for influencing the delivery of CTE services across 
secondary and postsecondary levels. 


First, the requirement that all local subgrantees must offer at least one POS is a major change in 
Perkins IV. Perkins IV describes POS as (1) incorporating secondary education and 
postsecondary education elements; (2) including coherent and rigorous content aligned with 
challenging academic standards and relevant CTE content in a coordinated, nonduplicative 
progression of courses that align secondary education with postsecondary education; (3) possibly 
including an opportunity for secondary students to participate in dual or concurrent enrollment 
programs or other ways to acquire postsecondary credits; and (4) leading to an industry- 
recognized credential or certificate at the postsecondary level, or to an associate or baccalaureate 
degree. 


The second new change is that accountability is strengthened in Perkins IV in several ways. 
Accountability for performance is extended from the state agency to local subgrantees, giving a 
state more leverage in holding its subgrantees responsible for improving the outcomes associated 
with their CTE programs. Local subgrantees (generally local education agencies [LEAs] at the 
secondary level and community colleges at the postsecondary level) have the option to negotiate 
with their state agency their level of expected performance, with those falling short of their 
performance benchmarks required to develop a program improvement plan. Continued failure to 
achieve negotiated targets could lead to the loss of some or all of their federal Perkins Act 
funding. 


Congress has required that states use their measures of academic achievement (in mathematics 


and reading or language arts) and of high school graduation required in the Elementary and 


* Dual or concurrent enrollment allows secondary students to earn college credits while also meeting requirements 
for high school graduation. This terminology is not consistent across jurisdictions, and such programs are 
implemented in various ways. 
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Secondary Education Act of 1965 as amended (ESEA) for assessing the performance of 
secondary CTE students.” States must report disaggregated performance data, and identify any 
gaps in performance compared to all CTE students, for certain categories (subgroups) of students 
specified in the ESEA as well as for the special populations defined in Perkins IV. Congress 
further required a number of other student performance measures, including indicators of student 
attainment of career and technical skill proficiencies such as student achievement on technical 
assessments, which are aligned with industry-recognized standards, if available and appropriate. 


The U.S. Department of Education has issued nonregulatory guidance that allows states to limit 
performance reporting to the subgroup of CTE students who are CTE concentrators. At the 
secondary level, a CTE concentrator is primarily a “secondary student who has earned three (3) 
or more credits in a single CTE program area (e.g., health care or business services). States are 
to report the percent of CTE concentrators who have passed academic requirements and 
“technical skill assessments that are aligned with industry-recognized standards, if available and 
appropriate.” At the postsecondary level a concentrator is primarily a student who completes at 
least 12 academic or CTE credits within a single program area sequence that is comprised of 12 
or more academic and technical credits and terminates in the award of an industry-recognized 
credential, a certificate, or a degree. 


The guidance further explains that the Department recognizes that a “State may not have 
technical skill assessments that are aligned with industry-recognized standards in every CTE 
program area and for every CTE concentrator.” In this regard, the State Plan guide for Perkins IV 
required each state to identify the program areas for which it had technical skill assessments and 
submit a plan and timeframe for increasing the coverage of programs and students reported in 
this indicator to cover all CTE concentrators and all program areas in the future. 


The final report will provide a careful examination of the state-reported performance information 
in response to Perkins IV. Also the final report will examine how states have interpreted the 
provision “if available and appropriate” for reporting students’ technical skills, including the 
consistency of reporting across states. Chapter 3 of this interim report on secondary CTE 
participation will assess the implications of limiting secondary performance reporting to CTE 
concentrators. 


Third, although Perkins IV allows some flexibility in how its funds are spent, the purposes of the 
law provide some specific guidance for funding uses. One new focus is on preparing students for 
“high-skill, high-wage, or high-demand occupations in current or emerging professions. Another 
focus is on providing professional development for improving leadership and the quality of CTE 
personnel at all levels (teachers, faculty, administrators, and counselors). And third, giving states 
the flexibility to consolidate their Title IJ (Tech Prep) funds into their Title I (Basic State Grants) 
funds is important because it allows states to tailor their CTE programs to their unique 
circumstances. These funding uses will be examined in the final report. 


> Perkins IV does not define a CTE student. However, the states have adopted Perkins IV performance measures that 
cover students concentrating in CTE. Although the definition of CTE concentrator varies across states, at the 
secondary level it generally includes students who take at least three occupational courses in one occupational area, 
unless only two courses are offered in an area, and at the postsecondary level it includes students who complete at 
least 12 academic or CTE credits in a program that consists of 12 or more credits, or who complete a shorter CTE 
program, that terminates in an industry-recognized credential, certificate, or degree. 
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This program logic model guides NACTE’s studies for examining the ways states and local 
entities are implementing Perkins IV. However, Perkins IV has only been in effect for a short 
period of time. Perkins IV was enacted in 2006, and states and local entities were allowed school 
year 2007—08 as a transition year before fully implementing the new law in school year 2008-09. 
Hence, it will be the final report that will explore most of the initial effects of Perkins IV, with 
the interim report findings establishing a baseline by which to evaluate Perkins IV. 


ES-3 Chapter 3: Participation in Secondary Career and Technical Education from 
1982 to 2004 


Chapter 3 examines changes in participation in secondary CTE among high school graduates 
from 1982 to 2004, the period from before the Carl D. Perkins Vocational Education Act 
(Perkins I) was enacted in 1984 into the period covered by the Carl D. Perkins Vocational and 
Technical Education Act of 1998 (Perkins III). Over this period, CTE moved away from being 
what many described as a high school curriculum track—then known as vocational education, in 
contrast to academic and general tracks—students focused on occupationally specific courses in 
one occupational area. Instead, CTE has shifted to an educational field, in which students spend 
relatively less time in one occupational area but are more likely to explore several. 


The background for the changes in CTE participation is the shift in which a majority of 
secondary graduates complete the academic curricular foundation defined by the new basics 
courses recommended in the 1983 report, A Nation at Risk (National Commission on Excellence 
in Education, 1983). These new basics consist of at least four Carnegie credits in English and 
three credits each in mathematics, science, and social studies. The proportion of high school 
graduates choosing the academic foundation consisting of the new basics rose from 14 percent in 
1982 to 60 percent in 2004 (Dalton et al., 2013). 


As the shift to more academic coursetaking occurred, patterns of investing in secondary CTE 
occupational courses changed® (Exhibit ES-1): 


¢ While the percentage of graduates enrolling in at least one CTE course declined by a 
small amount, almost all high school students (96 percent in 2004) still enroll in at least 
one CTE course, so that CTE has some meaning for nearly all graduates. 


¢ The share of graduates who are CTE investors—those who earn three or more 
occupational credits—declined from 46 percent in 1982 to 35 percent in 1992, but that 
share has since stabilized and increased between 1992 and 2004 from 35 percent to 
38 percent of high school graduates. 


¢ A key measure of CTE occupational concentration is earning three or more credits of 
CTE in a single occupational area—there was a decline in the percent of occupational 
concentrators from 30 percent of all graduates in 1982 to 17 percent in 2004. 


° All differences and changes over time described in the executive summary are statistically significant at p<.05, 
unless otherwise noted. 
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* However, another key descriptor of CTE investors is the percent of high school graduates 
who took three or more CTE credits in more than one occupational area (CTE explorers). 
The proportion of CTE explorers increased from 16 percent in 1982 to 21 percent in 
2004. 


Exhibit ES-1. Percentage of high school graduates participating in career and technical 
education, by level of CTE occupational coursetaking: 1982, 1992, 2004 
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Exhibit reads: The percentage of public high school graduates with any CTE credit was 98 percent in 1982. 


NOTE: CTE = career and technical education. CTE concentrators earned three or more credits in at least one occupational area. 
CTE investors consist of CTE concentrators plus CTE explorers who earn three or more total occupational credits but did not 
concentrate in an occupational area. A Carnegie unit is equivalent to a course taken for one period each day for one full school year. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


The fact that there are currently more students who invest in CTE who are explorers (three or 
more credits in more than one occupational area) rather than concentrators (three or more credits 
in One occupational area) has important implications for performance reporting under Perkins IV. 
Federal nonregulatory guidance, as noted above, allows states to report performance data for 
concentrators and hence, states are permitted to report on less than half of investors who take 
three or more CTE courses upon graduation. 


As the share of graduates investing in three or more CTE credits stabilized between 1992 and 
2004, CTE investors were also able to close nearly all the gaps in the rate of completing “new 
basic” courses compared with non-investors in CTE (Exhibit ES-2): 


* In 1982, only about 6 percent of CTE investors completed the new basics, compared with 
about 22 percent of other students. 
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¢ By 2004, about 58 percent of CTE investors had completed the new basics, compared 
with about 61 percent of other students. 


Exhibit ES-2. Percentage of CTE investors and other high school graduates (non-investors), 
taking the new basics courses: 1982, 1992, 2004 
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Exhibit reads: In 1982, 6 percent of CTE investors completed the new basics courses. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors are all other students—those who earned zero to less than three total 
occupational credits (Carnegie units). A credit (Carnegie unit) is equivalent to a course taken for one period each day for one full 
school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


Students could take CTE and still complete the new basics course load at greater rates because of 
the increase in total credits earned. The total number of credits earned by high school graduates 
went from 21.7 credits in 1982 to 26.2 credits in 2004, an increase of over one-fifth. 


There was also major narrowing in rates of postsecondary attendance between CTE investors 
and non-investors. Between 1982 and 2004 high school graduates who were CTE investors 
significantly increased their postsecondary attendance during their first two years after high 
school. Among CTE investors, the proportion of CTE investors attending postsecondary 
programs increased from 54 percent in 1982 to 75 percent in 2004, narrowing the postsecondary 
rate of attendance gap with non-investors from 23 percentage points to only 8 percentage points. 


ES-4 Chapter 4: Outcomes of Career and Technical Education: Some New 
Evidence 


The mandate for the NACTE in Perkins IV includes not only examining the law’s 
implementation but also analyzing the impact CTE has on academic achievement and 
employment outcomes. 
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NACTE’s investigations into the impact of CTE on outcomes extend the findings from the 
earlier mandated study of Perkins III, namely, the National Assessment of Vocational Education 
(NAVE; Silverberg et al., 2004), which largely examined CTE outcomes occurring under 
Perkins II. At the secondary level, NAVE reported that research evidence showed that CTE did 
not bring about or contribute to academic achievement gains. NAVE also reported that only 
mixed evidence exists that CTE reduces the high school dropout rate. Moreover, it reported, CTE 
neither helped nor hurt the chances of students going on to postsecondary schooling. 
Postsecondary vocational education did, however, provide significant economic returns, with 
those earning a credential (certificate or degree) experiencing the largest benefits. 


The NACTE looked for rigorous new evidence of CTE outcomes and found opportunities using 
existing data: a national study of secondary students using the NCES ELS:2002 and a natural 
experiment involving choice of a CTE high school in Philadelphia. The CTE outcomes studied 
occurred in periods covered by Perkins II and Perkins III. 


The study of ELS:2002 looked at a representative sample of high school sophomores in 2002 and 
examined the relationship of CTE coursetaking to math achievement gains and dropping out of 
high school in 2004. In particular, this study examined differences in outcomes for students who 
take more versus fewer CTE courses in traditional high schools. NACTE’s study using 
ELS:2002 (Bozick and Dalton, 2013) analyzes gains in math using fixed effects regression 
analysis that reduces the influence of unmeasured factors—such as attitudes, preferences, or 
innate ability—that may lead to self-selection into certain educational programs. 


The ELS:2002 study failed to find any relationship between CTE coursetaking and student 
outcomes (Bozick and Dalton, 2013): 


¢ The total number of occupational credits earned during the last two years of high school 
had no relationship to the increase in number of correct answers on the mathematics 
assessment. 


* Controlling for socioeconomic and academic differences among students as well as 
semester timing, the number of credits students earned in occupational courses was 
unrelated to their likelihood of dropping out. 


A second study took advantage of a natural experiment in Philadelphia involving the choice of 
CTE high schools. Within the context of the district’s high school choice program, more students 
applied for entry to Philadelphia’s five CTE high schools than could be accepted, and acceptance 
was based on a random lottery. Therefore, the study examined the effects of winning admission 
to a CTE high school as opposed to attending another type of high school where they might also 
take CTE courses. This permitted comparing the outcomes of students accepted into CTE high 
schools with those who were not accepted. Outcomes for three cohorts of students were 
examined—the high school classes of 2003, 2004, and 2005. For two of the three cohorts (the 
classes of 2004 and 2005), the results are generalizable only to a more select group of 
Philadelphia students who applied to a CTE school and passed some moderate screens for 
achievement, attendance, and behavior. 
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The natural experiment in choice of CTE high schools in Philadelphia found some positive 
effects from attendance at CTE schools: 


¢ Attending a CTE high school increased the probability of students successfully 
completing the college preparatory mathematics sequence of algebra 1, algebra 2, and 
geometry. 


¢ Attending a CTE high school had no relationship to math or reading achievement growth 
from eighth to 11th grades. 


¢ CTE high school attendance was related to higher on-time graduation rates. 


The findings from the ELS:2002 study of student outcomes support the prior NAVE findings that 
CTE courestaking does not contribute to academic achievement gains as measured by 
standardized achievement tests. However, evidence from a natural experiment in Philadelphia 
found a positive relationship between attending a CTE school and mathematics coursetaking and 
graduation. To further advance understanding of the potential effects of secondary CTE, 
evaluations would benefit from going beyond the approach of current studies that examine the 
outcomes of CTE without taking into account program variation or implementation quality. A 
more useful approach would be to focus on rigorously assessing the potential for CTE benefits 
from a defined and well-implemented set of programs. 


ES-5 Chapter 5: Career and Technical Education in International Perspective 


Chapter 5 provides international comparative information on CTE—more widely known 
internationally as vocational education—in 15 nations that are competitors to the United States 
and participants in data collections sponsored by OECD. International comparisons address one 
of the legislative purposes of Perkins IV: to provide students the opportunities to develop “the 
knowledge and skills needed to keep the United States competitive.” The Congress has also 
expressed interest in international comparisons under section 114(a)(3), which provides that 
NCES “may include international comparisons” in its assessments of CTE. Moreover, major 
education organizations including the National Governors Association and the Council of Chief 
State School Officers have also called for international benchmarking studies to help maintain 
and improve the competitiveness of the United States.’ 


OECD data show that secondary students in competitor nations devote a higher share of their 
course work to secondary vocational education programs than the United States, with the 
exception of Canada (see Exhibit ES-3). Using OECD standards of vocational education 
enrollment, in most European countries examined, almost half or more of secondary school 
graduates enroll in a vocational-oriented program (in which they earn 25 percent or more of their 
total credits). Even in Korea and Japan, with their strong academic traditions, a quarter of 
secondary graduates concentrate in vocational education by OECD standards. By contrast, since 
the early 1980s the percentage of U.S. secondary students meeting OECD standards declined 
from about 18 percent to 6 percent. 


7 National Governors Association, Council for Chief State School Officers and Achieve. Benchmarking for Success: 
Ensuring U.S. Students Receive a World-Class Education. 
http://www.achieve.org/files/BenchmarkingforSuccess.pdf. 
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Exhibit ES-3. Percentage of upper secondary school enrollment in concentrated vocational 
education (25 percent of total credits) in OECD countries: 2006 
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Exhibit reads: The percentage of upper secondary students enrolled in concentrated vocational education is 78 
percent for Austria. 


SOURCE: Organization for Economic Co-operation and Development (OECD), Education at a Glance 2008 (2008, Table C1.1). 
Website: http:/Awww.oecd.org/edu/eag2008. 


It is worth noting that 25 percent of total credits earned in CTE in the United States is well above 
the definition of three CTE credits in one occupational area used to define a CTE concentrator 
for Perkins IV accountability reporting. The OECD definition of a vocational education 
concentration would be equivalent to five or six CTE credits in the United States, which is more 
typical of U.S. career academy models or programs with an apprenticeship or internship 
component. 


ES-6 Chapter 6: Implications and Next Steps 


Evidence from the interim report suggests that secondary CTE is shifting away from its historical 
dual-track approach and that successive cohorts of secondary CTE students are closing the gap 
with non-CTE students with respect to the number of core academic credits earned and college 
going rates. The interim report findings about CTE participation, effectiveness, and international 
benchmarking, also raise important implementation and outcome issues that warrant further 
exploration in the final NACTE report and beyond. These issues include: 


* How to assess the outcomes of the increasing proportion of students who invest in CTE 
courses (take three or more credits) but take fewer than three credits in a single 
occupational area (explorers). CTE explorers are not part of the current Perkins 
performance accountability system yet they are a growing majority of CTE investors. 
Research could examine the implications of including the category to assess program 
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performance. Given that explorers do not concentrate in any one CTE field, the program 
questions to address are what are the expected outcomes of explorers and how could 
these outcomes be assessed within the accountability system? 


* How to reconcile different results from studies about the benefits of secondary CTE on 
academic coursetaking, achievement, and high school graduation? Some studies find no 
such benefits from CTE, but other studies find CTE strengthening academic coursetaking 
and graduation rates. Research could explore whether these differences arise from 
differences in study methodology, nature of CTE courses and programs, or differences in 
CTE population and context. Research could also move beyond only exploring average 
effects of CTE without taking into account program variation or implementation quality 
and focus on assessing the benefits from a set of defined and well-implemented 
programs. 


The first question about CTE accountability coverage and measures will be examined closely in 
the final report. The second question about the empirical benefits of secondary CTE populations 
will be difficult to address given present data. With about half of all secondary students not 
going directly to college, and many having difficulties finding employment, it is important that 
information on the effectiveness of potentially exemplary CTE be investigated. 


There are two reasons for creating an evaluation agenda of studies that examine specific CTE 
programs or interventions in the near future. First, existing evidence shows the effectiveness of 
programs like career academies (e.g., Kemple and Willner, 2008). While these programs are now 
widely offered, much is unknown about the variation among them and what that variation might 
mean for program outcomes and their effectiveness in turning around whole high schools. 
Following students from well-implemented programs could be especially informative for 
identifying what works in improving student outcomes. 


A second reason for initiating new studies of CTE programs is that OECD data suggest that 
many of our global competitors having substantial shares of secondary students enrolled in more 
CTE programs also have higher achievement on international tests like the Program for 
International Student Assessment and higher secondary completion rates. This suggests, and 
rigorous studies need to confirm, that it is feasible to provide a sound academic and additional 
technical education at the secondary level. 


In conclusion, the transformation of CTE culminating in Perkins IV represents key program 
innovations to integrate academic and technical skills through programs of study and new 
accountability systems. The success of these innovations will depend upon sound 
implementation and having access to rigorous and timely evaluative data on what works and 
what is not working. The evidence in the interim and final NACTE reports along with 
implementing the proposed research agenda will help provide the needed evaluative data. 
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Exhibit ES-4. Perkins 1V requirements for NACTE and associated NACTE studies 


Pursuant to section 114(d)(2)(B) of Perkins IV, 
Legislative Requirements: “[t]he assessment 
required ... shall include descriptions and 
evaluations of— 


“(i) the extent to which State, local, and tribal entities 
have developed, implemented, or improved State 
and local career and technical education 
programs assisted under the Act; 


“(ii) the preparation and qualifications of teachers and 
faculty of career and technical education (such as 
meeting State established teacher certification or 
licensing requirements), as well as shortages of 
such teachers and faculty; 


“(iii) academic and career and technical education 
achievement and employment outcomes of 
career and technical education, including 
analyses of— 

(I) the extent and success of the integration of 
rigorous and challenging academic and 
career and technical education for students 
participating in career and technical 
education programs, including a review of the 
effect of such integration on the academic 
and technical proficiency achievement of 
such students (including the number of such 
students receiving a secondary school 
diploma); and 

See notes at end of table. 


NACTE studies 


e Accountability and programs of study 
Surveys of state career and technical education 
(CTE) directors and local Perkins funds subrecipients 
(secondary and postsecondary) 

— Case studies of states, districts, schools, and 

postsecondary institutions 

— Expert panel review of state programs of study 

— Analysis of Perkins performance data systems 
e Funding 

— State CTE director survey 


e Tech Prep 
— State CTE director survey 
e Promising practices 
— Literature review of effectiveness of online 
technology and distance education in CTE 
— Developing comprehensive, longitudinal state 
databases for CTE reporting 
e Teachers and faculty 
— Secondary analysis of National Center for Education 
Statistics (NCES) data and reports 
— Analysis of SAT scores of CTE teachers 
e Supply and demand 
— Secondary analysis of data from NCES and Higher 
Education Opportunity Act Title II 
e Integration of academic/career and technical 
education content 
— Surveys of state CTE directors and local Perkins 
funds subrecipients (secondary and postsecondary) 
— Expert panel review of state programs of study 
— Case studies of selected states, districts, schools and 
postsecondary institutions 


Continues next page 
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Exhibit ES-4. 


Pursuant to section 114(d)(2)(B) of Perkins IV, 
Legislative Requirements: “[t]he assessment 
required ... shall include descriptions and 
evaluations of— 


(Il) the extent to which career and technical ° 
education programs prepare students, 
including special populations, for subsequent 
employment in high skill, high wage 
occupations (including those in which 
mathematics and science skills are critical), 
or for participation in postsecondary 
education; 


“(iv) employer involvement in, and satisfaction with, ° 
career and technical education programs and 
career and technical education students’ 
preparation for employment; 


“(v) the participation of students in career and e 
technical education programs; 


“(vi) the use of educational technology and distance ° 


learning with respect to career and technical 
education and tech prep programs; and 


“(vii) the effect of State and local adjusted levels of e 


performance on the delivery of career and 
technical education services, including the 
percentage of career and technical education and 
tech prep students meeting the adjusted levels of 
performance described in section 113.” 


Perkins IV requirements for NACTE and associated NACTE Studies—continued 


NACTE studies 


Impact and educational/occupational outcomes 

— Secondary analysis of national NCES longitudinal 
data Education Longitudinal Study of 2002 
(ELS:2002) 

— Secondary analysis (natural experiments and quasi- 
experiments) using local administrative data in 
Philadelphia and San Diego and state administrative 
data from Florida 


Employer involvement and satisfaction 
— Review of employer surveys 


Student participation 

— Secondary analysis of NCES data and reports (e.g., 
ELS:2002, High School and Beyond Longitudinal 
Study [HS&B], and National Education Longitudinal 
Study of 1988 [NELS:88]) 

— Secondary analysis of Organization for Economic Co- 
operation and Development (OECD) data on CTE 
participation 

Effectiveness of technology/distance education 

— Literature review of effectiveness of online 
technology and distance education in secondary CTE 


Adjusted levels of performance (also see 

Accountability above) 

— Surveys of state CTE directors and local Perkins 
funds subrecipients (secondary and postsecondary) 
Case studies of selected states, districts, schools and 
postsecondary institutions 


NOTE: NACTE = National Assessment of Career and Technical Education; Perkins |V = Carl D. Perkins Career and Technical 
Education Act of 2006 (20 U.S.C. 2301 et seq. as amended by P.L. 109-270). 
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Chapter 1. 


Introduction 


The Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) reinforces a 
longstanding federal commitment to supporting career and technical education (CTE). Although 
federal contributions to the CTE enterprise account for only a fraction of state and local 
spending, federal policy has had, and continues to have, a large influence on state and local 
programs and policies. 


Over time, as national attention has turned to globalization and its implications for a more highly 
skilled workforce, so too has the Perkins Act’s emphasis, with current legislation aimed at raising 
the academic and technical rigor of secondary and postsecondary CTE instruction to prepare 
students for entry into high-skill, high-wage, or high-demand occupations. New legislative 
provisions, which promote the alignment of secondary and postsecondary education through 
programs of study (POS) that, among other things, lead to an industry-recognized credential or 
certificate or degree at the postsecondary level, reinforce this goal. These are backed by 
secondary and postsecondary accountability provisions for student results. 


Under Perkins IV, Congress mandated a National Assessment of Career and Technical Education 
(NACTE) to evaluate the implementation of the law and its outcomes. This interim report is the 
first of two reports. The purpose of this NACTE interim report is to describe the research 
approach and present findings to date from the national assessment. This interim report consists 
of six chapters plus an appendix on the NACTE requirements. The first chapter® provides an 
introduction to Perkins IV and the NACTE research agenda, while the second chapter offers a 
framework for evaluating Perkins IV implementation. The remaining four chapters present 
baseline information focused on secondary CTE including participation trends, new outcome 
evidence and the international comparative information. 


1.1. Key Provisions in Perkins IV 


Federal legislation addresses particular challenges facing the nation, and these policy issues can 
and do change over time (Exhibit 1). During the early years of vocational education (1917-68) 
the stress was on expanding vocational programs to provide a trained semiskilled workforce to 
meet the needs of industry. Between 1968 and 1990, ensuring equal access to vocational 
education for disadvantaged and special need populations was stressed. Most recently, the stress 
is on improving the academic and technical quality of programs in the form of CTE along with 
strengthened performance standards to ensure program quality. 


* This chapter was prepared by ED staffers Jay Noell (retired) and Michael Fong with assistance from Steven Klein, 
MPR Associates. 
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Exhibit 1. Overview of changes in federal legislation of CTE 


Periods of CTE 
legislation Policy objectives and tools 
1917-63 Provide trained workers for growing semiskilled occupations and retain more 
students in secondary education through: 
e Expansion of separate vocational schools and programs. 
e Funds for basic maintenance of programs. 
e Focusing on agriculture, industry, and home economics for high school 
students. 


1963-68 Improve and expand vocational education through: 

e Separate funds for innovative programs, research, and curriculum 
development. 

e Support for construction of regional or area vocational schools. 

e Support for adult training and retraining (postsecondary vocational 
education). 

e Encouragement to states to promote vocational education equity, better 
service to special populations. 


1968-90 Improve vocational education and facilitate access through: 
e Periodic encouragement to states to distribute some funds by community's 
economic need and levels of student disadvantage. 
e Establishment and expansion of set-aside funds to serve special 
population groups. 
e Prohibiting the use of most federal funds for maintenance of programs. 
e Continuation of set-aside funds for program improvement. 


1990-98 Expand equal access and emphasize academic quality through: 

e Introducing intrastate and intradistrict funding formulas: distribution to 
agencies and schools weighted by special populations. 

e Promoting integration of academic and vocational education and “all 
aspects of the industry.” 

e Set-aside funds for new program linking secondary and postsecondary 
vocational education: Tech Prep. 

e Requirement that states develop performance standards. 


1998-Present Improve academic and technical quality in CTE programs through: 

e Introducing programs of study, a sequence of high school and college 
course work that combines academic and career-related courses that lead 
to a credential. 

e Requirement that states and local subgrantees develop performance 
standards around core academic, technical, and graduation/placement 
indicators. 

e Elimination of most set-asides but greater share of funds and flexibility to 
local subgrantees in exchange for an accountability system that would be 
used to improve student performance and document gaps in performance 
by student subgroups. 


NOTE: CTE = career and technical education. 
SOURCE: Millsap and Muraskin (1992); Boesel et al. (1994). 
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Perkins IV employs three major avenues for improving the academic and technical quality of 
CTE services: 


* Programs of study (POS), defining the expectations of a comprehensive sequence of CTE 
instruction; 


* Accountability provisions, holding state agencies and local subgrantees responsible for 
their performance; and 


¢ Spending directives, detailing the purposes and required uses of federal funds. 


These three thematic areas provide a range of policy levers for influencing the delivery of CTE 
services across secondary and postsecondary levels. 


First, the requirement that all local subgrantees offer at least one POS is perhaps the most 
ambitious change in Perkins IV. Perkins IV describes POS as (1) incorporating secondary 
education and postsecondary education elements; (2) including coherent and rigorous content 
aligned with challenging academic standards and relevant CTE content in a coordinated, 
nonduplicative progression of courses that align secondary education with postsecondary 
education; (3) possibly including an opportunity for secondary students to participate in dual or 
concurrent enrollment programs or other ways to acquire postsecondary credits;’ and (4) leading 
to an industry-recognized credential or certificate at the postsecondary level, or to an associate or 
baccalaureate degree. This direction aligns well with recent initiatives for better integrating 
secondary and postsecondary education. For example, some states have linked K-12 and 
postsecondary data systems. 


Second, accountability is strengthened in Perkins IV in several ways. Notably, accountability for 
performance is extended from the state agency to local subgrantees, giving states more leverage 
in holding their subgrantees, which generally are secondary local education agencies (LEAs) and 
community colleges, responsible for improving the outcomes associated with their CTE 
programs. Local subgrantees (which are referred to as “eligible recipients” in Perkins IV and are 
generally LEAs at the secondary level and community colleges at the postsecondary level) have 
the option to negotiate with their state agency their level of performance or targets. Those falling 
short of their performance levels are required to develop a program improvement plan. 
Continued failure to achieve negotiated targets could lead to the loss of some or all of their 
Perkins IV funding. 


Congress also has required that states use the measures of academic achievement (in 
mathematics and reading or language arts) and of high school graduation required in the 
Elementary and Secondary Education Act of 1965 as amended (ESEA), for assessing the 


* Dual or concurrent enrollment allows secondary students to earn college credits while also meeting requirements 
for high school graduation. This terminology is not consistent across jurisdictions, and such programs are 
implemented in various ways. 
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academic performance of secondary CTE students.” States must report disaggregated 
performance data, and identify any gaps in performance compared to all CTE students, for 
certain categories (subgroups) of students specified in ESEA as well as for the so-called special 
populations defined in Perkins IV. Congress further introduced the accountability requirement 
that states report separately on programmatic outcomes for students participating in Tech Prep 
programs funded under Title II of Perkins IV, although no level of performance must be 
stipulated for these measures." 


Third, although Perkins IV allows some flexibility in how states and local entities spend their 
funds, the purposes of the law provide specific spending directives for funding uses. A new 
spending directive is on preparing students for “high-skill, high-wage, or high-demand 
occupations in current or emerging professions.” This purpose reflects congressional awareness 
that technological changes along with shifts in global markets are driving occupational changes 
in the American workforce. Another is on providing technical assistance for improving 
leadership and the quality of CTE personnel at all levels (teachers, faculty, administrators, and 
counselors). Leadership is a key responsibility in a time of change, and the quality of CTE staff 
is critical for preparing qualified students. States are also given the flexibility to consolidate their 
Title If Tech Prep funds with their Title I Basic State Grants funds, which is important because it 
could allow states to tailor their CTE programs to their unique circumstances. Finally, another 
spending focus in Perkins IV calls for— 


Providing individuals with opportunities throughout their lifetimes to develop, in 
conjunction with other education and training programs, the knowledge and skills 
needed to keep the United States competitive [italics added]. 


This focus recognizes the challenges the United States faces in remaining competitive in a global 
economy, and that individuals may need to continue their education and training throughout their 
lifetimes to remain active members of the workforce. 


1.2 National Assessment of CTE 


Perkins IV directs the secretary of ducation to conduct “an independent evaluation and 
assessment of career and technical education programs under the Act, including the 
implementation of the [Act].” Following the enactment of the new law, the secretary of 
education directed the Office of Planning, Evaluation and Policy Development (OPEPD) to 
conduct the NACTE. 


'° Perkins IV does not define a CTE student. However, the states have adopted Perkins IV performance measures 
that cover students concentrating in CTE. Although the definition of CTE concentrator varies across states, at the 
secondary level it generally includes students who take at least three occupational courses in one occupational area, 
unless only two courses are offered in an area, and at the postsecondary level it includes students who complete at 
least 12 academic or CTE credits in a program that consists of 12 or more credits, or completes a shorter CTE 
program, that terminates in an industry-recognized credential, certificate, or degree. 


'' See next paragraph: Perkins IV also allows funds for Tech Prep programs under Title II to be consolidated with 
funds for programs under Title I, eliminating separate reporting requirements, and about half the states have done so. 
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Congressional Mandate 


In its mandate (see Appendix A), Congress specified that NACTE should address issues that 
include the following: 


* participation of students in CTE programs; 

* academic and CTE achievement and employment outcomes of CTE; 

¢ funding distribution and uses; 

* effect of accountability provisions on the delivery of CTE services; and 


* extent to which state and local entities have developed, implemented, or improved state 
and local CTE programs assisted under the Perkins Act. 


The research NACTE conducts is to be reported in interim and final reports. This interim report 
will address parts of the first and second issues above: the participation of students in secondary 
CTE programs and a first look at the academic outcomes of CTE from two of NACTE’s 
quantitative studies. The final report will address the other topics as well as those listed in 
Appendix A. 


Evaluation Issues and Strategies 


The primary policy question is whether at both secondary and postsecondary levels CTE is 
effective in helping students attain the academic and technical skills required for securing high 
skill, high wage, and high demand jobs, or put more broadly, in preparing students for college 
and careers. Perkins IV attempts to strengthen CTE and improve its quality by increasing the 
emphasis on accountability and introducing POS that integrate and align, within and across 
secondary and postsecondary levels, challenging and rigorous academic and technical content as 
conditions for federal funding. 


NACTE’s evaluation agenda examines the implementation of provisions in Perkins IV to 
strengthen CTE and the outcomes of CTE through a set of studies addressing the congressionally 
mandated issues identified above, as follows— 


1. How have secondary and postsecondary student enrollments in CTE programs 
changed? Examples of studies providing data for the NACTE include secondary data 
analysis of the National Center for Education Statistics (NCES) longitudinal data from 
the Education Longitudinal Study of 2002 (ELS:2002), the National Education 
Longitudinal Study (NELS:88), the High School and Beyond (HS&B) study, the National 
Assessment of Educational Progress (NAEP) High School Transcript Studies, and the 
Integrated Postsecondary Education Data System (IPEDS). 


2. Do students who participate in CTE realize any educational or workforce benefits? 
NACTE studies include secondary data analysis of ELS:2002 and secondary data 
analysis of natural experiments and quasi-experiments using local administrative data in 
Philadelphia and San Diego and state administrative data from Florida. 
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3. How are Perkins IV funds distributed and used? NACTE studies include analysis of 
subgrantee fiscal data; state, district, and postsecondary institution surveys; and case 
studies of selected states, districts, schools, and postsecondary institutions. 


4. To what extent are Perkins IV accountability data valid, reliable, and comparable, 
and to what extent are decision-makers using these data? NACTE studies include 
state, district, and postsecondary institution surveys; case studies of selected states, 
districts, schools, and postsecondary institutions; and analysis of Perkins IV performance 
data systems. 


5. Are grantees meeting the requirement to develop and implement programs of study 
(POS) that integrate academic and technical content? NACTE studies include state, 
district, and postsecondary institution surveys; expert panel review of state POS; and case 
studies of selected states, districts, schools, and postsecondary institutions. 


Thus, NACTE is sponsoring a set of interrelated but distinct studies to address the congressional 
requirements and associated research questions. The overall agenda calls for diverse data 
collection and analytic methods: qualitative case studies, national surveys of states and local 
entities, use of national longitudinal and cross-sectional databases, use of state and local 
administrative databases, an expert panel, and both a natural experiment in choice of CTE 
schools and econometric estimation of national and local program effects. 


The short time period available to NACTE for conducting its research and reporting findings 
affects the ways evaluation studies can be carried out and how the data they produce may be 
used. Although NACTE is collecting new data focused on the state and local implementation of 
Perkins IV, available data largely capture outcomes associated with the periods in which 
Perkins IJ—the Carl D. Perkins Vocational and Applied Technology Education Act of 1990— 
and Perkins I[]—the Carl D. Perkins Vocational and Technical Education Act of 1998—were in 
effect. The interim report is relying on available data which do not reflect Perkins IV; however, 
the final report will include findings from new data collections under Perkins IV. 


NACTE’s surveys of state and local entities for the final report will assess Perkins IV 
implementation for program year 2008—09. The data collected will capture the early 
implementation of Perkins IV at the state and local levels but may not reflect the longer term 
effects of Perkins IV. 


Independent Advisory Panel 


Congress further mandated that an Independent Advisory Panel (IAP) be formed for NACTE. 
The purposes of the IAP are to— 


* Provide guidance to the secretary of education on the implementation of the NACTE, 
including the issues to be addressed and the methodology of its research studies to ensure 
that they meet the highest standards of quality. 


¢ Prepare an independent analysis of the findings and recommendations resulting from the 
assessment. 
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The U.S. Department of Education has assembled an IAP whose members have a wide range of 
expertise (see p. 1x for a list of the members). 


Interim Report: A Special Focus on Secondary CTE 


The findings in this interim report focus primarily on the role of secondary CTE in the 
preparation and transition of youths for college and careers. While postsecondary CTE plays a 
critical role in workforce development and specialized skills training, the final report will 
examine this area more fully when the NACTE studies on postsecondary education have been 
completed. 


For a large part of the 20th century, CTE (formerly vocational education) was a mainstay in the 
career readiness preparation of the majority of secondary students who entered the labor force 
directly from high school. Those going on to college were schooled in the academic track and 
exposed to a more rigorous set of college preparation courses. Vocational majors in secondary 
school were placed on a track that was more occupationally focused. As greater percentages of 
high school graduates enrolled in college directly from high school, this dual track system 
resulted in increasingly negative perceptions of vocational education as programs in which 
students were not adequately prepared for further education at the postsecondary level. 
Moreover, employers raised their expectations that new job entrants would bring strong basic 
academic skills, including high school graduates who were firmly grounded in reading, writing 
and mathematics skills (Levy and Murnane, 2004). 


Congress responded over the years by moving to modernize the federal vocational education law 
to meet the newer and more rigorous education and workplace requirements and eliminate the 
sharp separation between the academic and CTE course work under the dual track approach. The 
strengthening of the integration of academic and technical tracks continued in Perkins IV. One 
key feature is the introduction of POS, which as noted, establish in different occupational areas 
formal course sequences connecting high school and college work. POS combine academic and 
career-related courses and lead to credentials or certificates of attainment. Perkins IV also 
requires states and subgrantees to report on performance standards around core academic, 
technical, and graduation and placement indicators. 


The strategy of integration between academic and technical content also seeks to prepare 
secondary CTE students for postsecondary education and careers. The movement over the years 
has been to strengthen secondary CTE students’ access to a sound academic program while 
maintaining a strong technical program. This should permit students access without wasteful 
remediation at the postsecondary level but also in building a foundation toward “high-skill, high- 
wage, or high-demand occupations in current or emerging profes sions.”!” 


A special challenge facing CTE in meeting these twin goals of academic and career preparation 
is the role of CTE for the approximately half the students in each secondary class who do not 
directly enter college upon leaving high school. These include the approximately one-third of the 


2 Perkins IV, S 250-2. 
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high school graduating class who currently do not directly go on to postsecondary education as 
well as students who drop out of high school before graduation.’ 


To gain a measure of the challenge in preparing students for the workforce, the employment of 
recent high school graduates and dropouts declined sharply between 2007 and 2011. Since the 
financial crisis, the labor force participation rate of high school graduates is down, the 
unemployment rate is up, and a majority of graduates not enrolled in college (54.3 percent) are 
jobless compared to 38.6 percent in 2007.'4 


The picture for high school dropouts is starker. Between 2007 and 2011, the unemployment rate 
for those looking for work has risen from 26.9 percent to 38.4 percent of high school dropouts. 
The jobless rate, which includes those who are not actively seeking work, went from 58.9 
percent in 2007 to 65.6 percent in 2011. 


To better understand the potential role of secondary CTE, this interim report will present 
information on the participation of secondary students in CTE using a variety of participation 
measures and examine the coursetaking and academic outcomes of secondary CTE students. The 
interim report will also compare CTE preparation in the U.S. to other industrialized nations. 


Organization of Interim Report 


This interim report provides baseline information and lays the groundwork for a comprehensive 
final report. It consists of six chapters: 


* Chapter 1 provides an overview of the NACTE research agenda. 


* Chapter 2 provides an introduction to the legislative provisions of Perkins IV and issues 
they raise for NACTE’s evaluation of the implementation and outcomes of the law. 


* Chapter 3 examines trends in student participation in CTE before Perkins IV was enacted 
using data from longitudinal studies of high school graduates conducted by the U.S. 
Department of Education’s NCES. 


* Chapter 4 summarizes prior studies of CTE outcomes and presents preliminary findings 
from two NACTE studies (a national study and a large city study) of secondary CTE 
outcomes associated with earlier versions of the Perkins Act. 


* Chapter 5 presents international benchmark data on pre—Perkins IV CTE participation 
and experiences comparing U.S. secondary students with their peers in other developed 
countries and drawing upon a report from the Organization for Economic Co-operation 
and Development (OECD, 2008). 


* Chapter 6 draws implications for the final report. 


'S Of the 3.1 million youths age 16 to 24 who graduated from high school in January through October 2011, 
2.1 million went directly on to college and one million (31.8 percent) were not enrolled in college in October 2011. 


'* Bureau of Labor Statistics. College enrollment and work activity of high school graduates news releases. Analysis 
of Current Population Survey data, 2007 and 2011. http://www.bls.gov/schedule/archives/all_nr.htm#HSGEC. 
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Chapter 2. 


Framework for Evaluating the Implementation of Perkins IV 
Provisions on Accountability and Programs of Study 


This chapter’> provides an introduction to the Carl D. Perkins Career and Technical Education 
Act of 2006 (Perkins IV) in the form of a logic model. Logic models are systematic and visual 
ways to depict how the elements of a program are supposed to work to bring about desired 
outcomes. 


The purpose of developing a logic model for Perkins IV is to guide the conduct of the 
independent evaluation and assessment of career and technical education (CTE) programs 
mandated by Congress in Perkins IV (see Section 114(d)(2) in Appendix A). The National 
Assessment of Career and Technical Education (NACTE) will provide information on the Act’s 
implementation in and influence on state and local programs. By identifying the key elements in 
the law, the sequences and causal assumptions about the way they operate and interact, and the 
incentives and requirements for required actions, the logic model provides direction for what 
NACTE should be examining in assessing whether Perkins IV is achieving its intended effects. 


2.1. Logic Model 


The underlying logic model presented in this chapter is premised on the interaction of two key 
legislative provisions contained in Perkins IV. First, it is postulated that programs of study (POS) 
demonstrating challenging academic standards and technical content, aligned across secondary 
and postsecondary levels, and offered in an articulated, nonduplicative sequence of courses, that 
lead to an industry-recognized credential or certificate, or a degree, will improve educational and 
career outcomes. Second, a valid and reliable system of accountability measures should provide 
overall information of CTE effectiveness and provide a basis for feedback and continuous 
improvement. The logic model (Exhibit 2) is organized in terms of critical inputs and activities 
as well as the student outcomes they are expected to yield. 


¢ System inputs, which include federal legislation and funding, supplemented by federal 
guidance and technical assistance and state and local policies, form the foundation upon 
which state and local program design and operation occur. 


¢ New activities in Perkins IV reflect congressional priorities in framing legislation that 
would more fully develop the academic knowledge and technical skills of secondary and 
postsecondary education of CTE students through the use of POS while holding systems 
accountable for both academic and technical preparation of CTE students. 


* Outcomes in Perkins IV are separated into school and workplace domains. School 
outcomes at the secondary level include academic outcomes that align with ESEA 
requirements, postsecondary placement rates and technical skills attainment. 
Postsecondary outcomes include retention rates and technical degrees or certificates. 
Workplace outcomes are employment and earnings. 


'S This chapter was prepared by Steven Klein, MPR Associates. 


National Assessment of Career and Technical Education: Interim Report 9 


yodey wieu| :uoeonpy jeoluyos | Pue 199I1eD JO JUaWISSEssY ;eUOeN 


Exhibit 2. 


Federal 
legislation, 
funding, 
nonregulatory 
guidance, and 
technical 
assistance 


State 
legislation, 
funding, 
policies, 
guidance, and 
technical 
assistance 


Local policy 
guidance, 
knowledge, 
technical 
assistance, 
and 
relationships 


Logic model of Perkins IV key reforms 


POS: College and Career 
Pathways 


Secondary and postsecondary 
elements 

Coherent and rigorous content 
aligned with challenging 
academic standards and 
relevant technical content 
Dual or concurrent enrollment 
Lead to industry-recognized 
credential or certificate at the 
postsecondary level, or toa 
degree 


Accountability Measures 
and Targets 
State and/or local 
benchmarks and targets 
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NOTE: ESEA = Elementary and Secondary Education Act of 1965 as amended; POS = programs of study. 
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The remainder of this discussion takes a closer look at the Perkins IV logic model with respect to 
the intended operation of four key elements: (1) policy guidance, (2) funding, (3) POS, and 
(4) accountability. 


Policy Guidance 


The U.S. Department of Education is responsible for assisting states in implementing the 
Perkins IV legislation. In keeping with past practice and with the limitation on federal 
regulations described in Sec. 318 of Perkins IV, the Department chose to issue nonregulatory 
guidance, rather than issue regulations for implementing the Act’s key provisions. For example, 
this nonregulatory guidance provides states with suggested approaches for defining CTE student 
populations and constructing performance measures. States have used this guidance in 
consultation with U.S. Department of Education staff to develop their own policy guidelines to 
support local programs in implementing the Act?’s provisions. 


Little is known about state strategies and local choices in implementing Perkins IV or how, and 
to what extent, federal nonregulatory guidance is filtered down to the local level. NACTE’s final 
report will examine this translation from statutory language to federal guidance to state policy to 
local implementation to understand how it influences the implementation of Perkins IV. 


Funding 


As in previous legislation, states have discretion in how federal funds are distributed between the 
secondary and postsecondary sectors, how reserve funds may be used to target resources for 
specified uses, and how the set-aside amounts are used to support nontraditional programming 
and to serve individuals in State institutions. 


One new provision in the Act is the option for states to consolidate their Title II (Tech Prep) 
funding with their Title I (Basic Grant) resources. States that choose to maintain Tech Prep as a 
separate funding stream continue to have flexibility in how funds are allocated and distributed to 
local consortia, because there is no statutory Tech Prep formula for making subgrants stipulated 
in the Act. Perkins IV also introduces requirements that could affect state and local uses of funds, 
in particular for POS design and developing accountability systems. 


Programs of Study (POS): College and Career Pathways 


Through the requirement that all local recipients of Perkins funds must offer at least one POS, 
Perkins IV provides a new programmatic strategy shifting away from isolated stand-alone CTE 
classes to requiring a coherent program. POS should be viewed as embodying challenging 
academic content and technical content conveyed in a progressive sequence of course work 
incorporating secondary and postsecondary education elements. Its adoption implies that 
adequate career preparation requires readiness for advanced learning beyond high school, and 
that CTE course work should culminate in the award of an industry-recognized credential or 
certificate at the postsecondary level, or to a degree. 


National Assessment of Career and Technical Education: Interim Report 11 


The federal legislation details four core elements that are essential to a POS: 
* incorporate secondary education and postsecondary education elements; 


* include coherent and rigorous content aligned with challenging academic standards and 
relevant CTE content in a coordinated, nonduplicative progression of courses that align 
secondary education with postsecondary education to adequately prepare students to 
succeed in postsecondary education; 


* may include the opportunity for secondary students to participate in dual or concurrent 
enrollment programs or other ways to acquire postsecondary education credits; and 


¢ lead to an industry-recognized credential or certificate at the postsecondary level, or an 
associate or baccalaureate degree. 


There are, however, a number of components within a Perkins-defined POS that may require 
additional attention. '° Though these components currently exist in many, if not most, CTE 
programs, state agency and local subgrantee staff have not consistently sought to integrate these 
parts into a unified instructional strategy. Components that are critical to the success of POS 
include the following: 


1. Secondary and Postsecondary Education Articulation 
¢ Agreement between the participating secondary and postsecondary Perkins subgrantees 
educators is reached that outlines the scope and sequence of content within the POS. 


¢ Joint adoption by the secondary and postsecondary subgrantees of rigorous technical 
content and the alignment of the delivery of content between the two educational sectors 
in a way that establishes a nonduplicative sequence of courses. 


¢ Identification by secondary and postsecondary subgrantees of relevant academic 
standards that support the desired outcomes for the POS. 


* Conscious attention by secondary and postsecondary subgrantees to constructing the 
secondary POS component to assure student attainment of sufficient knowledge and 
skills to avoid postsecondary remediation. 


2. Academic and Technical Content Alignment 
¢ Selection of relevant academic standards that align with secondary ESEA content 
standards and postsecondary certificate or degree general education requirements. 


* Selection of industry-based technical content for the POS area. 


¢ Selection or development of valid and reliable assessments of technical skill attainment. 


'© The federal legislation does not detail how the four core elements of POS should be operationalized in the field. 
Based on knowledge of state procedures, we examine each of the core elements to identify additional components 
that may need to be addressed in practice. It is important to emphasize that these are not mandated in the Perkins IV 
legislation. 
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* Curriculum, instruction, and assessments aligned with challenging academic and 
technical content. 


* Professional development for secondary and postsecondary instructors on offering and 
teaching courses in the POS. 


3. Dual or Concurrent Enrollment Accessibility 


¢ Implement dual or concurrent enrollment agreements that award high school students 
with required credits for a postsecondary certificate or degree. 


* Provide for integrating existing Tech Prep program components into POS. 


* Where feasible, co-locate joint secondary-postsecondary CTE instructional facilities or 
provide secondary student access to postsecondary CTE facilities when the student is 
ready for advanced postsecondary education within his or her POS. 


4. Credential, Certificate, or Degree Attainment 


¢ POS are designed with the intended outcome of an industry-recognized credential or 
certificate at the postsecondary level, or a degree that is recognized and valued by 
employers. 


¢ POS are constructed to promote preparation for career areas that are considered high 
skill, high wage, and high demand as defined by economic and workforce development 
needs and labor market information. 


The NACTE final report will explore in considerable detail these critical success factors in terms 
of how states and districts are implementing POS. It will explore the likelihood that state 
agencies and local subgrantees will adopt differing approaches to implementing the POS model. 
In some settings, many of the core elements and components of a POS may already exist in Tech 
Prep programs, and need only be integrated into the POS framework. Other agencies or 
subgrantees may need to create new strategies for administering and delivering program services, 
may discover additional components integral to POS support, or may find ways of merging 
elements to accomplish the indicated goal. In addition, the final report will examine the extent to 
which POS are available. Currently, each local CTE recipient is only required to implement one 
POS. 


Accountability Measures and Targets 


Congress used its 2006 Perkins reauthorization to clearly spell out a set of CTE performance 
indicators within educational levels. In a break from earlier legislation, policymakers crafted a 
separate set of performance measures for secondary and postsecondary education (Exhibit 3). 


They also introduced a new set of measures for Tech Prep programs, and extended accountability 
requirements to the local level by requiring that all subrecipients of Perkins IV funds negotiate 
local performance levels with their state CTE agency or accept their state’s statewide 
performance levels. 
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Exhibit 3. Performance indicators required by Perkins 1V and ESEA at the secondary and 
postsecondary levels 

Performance Perkins 

indicator, by level IV ESEA Performance indicator requirement 

Secondary Same as Yes Student attainment of challenging academic content standards 

1. Academic ESEA and student achievement standards, as adopted by a state 

attainment in in accordance with section 111(b)(1) of the ESEA and 
mathematics measured by state-determined proficient levels on academic 
assessments described in section 1111(b)(3) of the ESEA 

2. Academic Same as Yes Student attainment of challenging academic content standards 

attainment in ESEA and student achievement standards, as adopted by a state 

reading or in accordance with section 111(b)(1) of the ESEA and 

language arts measured by state-determined proficient levels on academic 
assessments described in section 1111(b)(3) of the ESEA 

3. Technical skill Yes No Student attainment of career and technical education (CTE) skill 

attainment proficiencies, including student achievement on technical 
assessments, that are aligned with industry-recognized 
standards, if available and appropriate 

4. Graduation rate Same as Yes Student graduation rates (as described in section 1111(b)(2)(C) 

ESEA of the ESEA) 

5. Completion rate Yes No Rates of attainment of a secondary school diploma, a GED 
credential or other state-recognized equivalent, anda 
proficiency credential in conjunction with secondary diploma, 
if offered in state 

6. Placement rate Yes No Placement in postsecondary education or advanced training, 
military service or employment 

7. Nontraditional Yes No Student participation in and completion of CTE programs that 

participation and lead to employment in nontraditional fields (defined as those 

completion for which one gender comprise less than 25 percent of the 
individuals employed in each such occupation or field of 
work) 

Postsecondary Yes — Student attainment of challenging career and technical skill 

1. Technical skill proficiencies, including student achievement on technical 

attainment assessments, that are aligned with industry-recognized 
standards, if available and appropriate 

2. Credential, Yes — Student attainment of an industry-recognized credential, a 

certificate, or certificate, or a degree 
degree attainment 

3. Retention or Yes — Student retention in postsecondary education or transfer to 

transfer baccalaureate degree program 

4. Placement Yes —_ Student placement in military service or apprenticeship program 
or placement or retention in employment 

5. Nontraditional Yes —_ Student participation in and completion of CTE programs that 


participation and 
completion 


lead to employment in nontraditional fields (defined as those 
for which one gender comprise less than 25 percent of the 
individuals employed in each such occupation or field of 
work) 


— Not applicable. ESEA is not applicable to the postsecondary level. 
NOTE: ESEA = Elementary and Secondary Education Act of 1965 as amended; Perkins IV = Carl D. Perkins Career and Technical 
Education Act of 2006 (20 U.S.C. 2301 et seq. as amended by P.L. 109-270). 
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To support states in refining their accountability systems under Perkins IV, the U.S. Department 
of Education issued nonregulatory guidance detailing preferred approaches for defining 
measurement populations and constructing performance measures. These guidelines included a 
description of the intent of each measure, a suggested approach for assessing student 
performance, and examples of numerator and denominator constructions. Implementing these 
accountability measures and guidelines raises a number of issues that NACTE will study 
including: 


Which CTE students are covered? Under federal guidelines states report the performance only 
of CTE concentrators. The nonbinding guidance encourages states to adopt the following 
definitions to identify students concentrating in CTE programs: 


Secondary CTE Concentrator. A student who has earned three or more credits in one 
CTE program area, or two credits in one CTE program area, but only in those program 
areas where two credit sequences are recognized by the state and/or its local eligible 
subrecipients. 


Postsecondary CTE Concentrator. A postsecondary/adult student who completes at least 
12 academic or CTE credits within a single program area sequence comprised of 12 or 
more academic and technical credits that terminates in the award of an industry- 
recognized credential, a certificate, or a degree, or completes a short-term program 
sequence of less than 12 credit units that terminates in an industry-recognized credential, 
certificate, or degree. Mr 


Because states have flexibility in adopting their definition of a CTE concentrator, Chapter 3 
examines the effect of reporting program performance using a variety of CTE participation 
measures. 


How accurate are performance indicators reported? Legislative requirements and U.S. 
Department of Education policy guidance appear to have focused state reporting around a 
common set of performance measures although a number of issues merit further study by the 
NACTE. One is that the Perkins IV does not adequately address the contribution that CTE can 
make to academic performance of high school graduates. Although states are now required to use 
their ESEA measure to assess the academic attainment of CTE concentrators, many states assess 
students in the spring of their 10th-grade or fall of their 11th-grade year, prior to when students 
complete their CTE concentration. Thus, the measure is more likely to reflect the attainment of 
those who enter CTE than the effects of participation in CTE. 


Perkins IV also does not allow for trade-offs among the performance indicators—all are assumed 
to be equally important and attainable. Measures are seen as independent of one another; that is, 
a state or local program that underperforms on one measure may not offset its shortfall with 
superior performance on another. Possible trade-offs in performance across indicators are also 


'’ Definitions cited are referenced in the U.S. Department of Education’s March 13, 2007, program memorandum, 
entitled Student Definitions and Measurement Approaches for the Core Indicators of Performance Under the Carl 
D. Perkins Career and Technical Education Act of 2006 (Perkins IV). A copy of the document may be accessed at 
http://www.ed.gov/about/offices/list/ovae/pi/memoperkinsiv.html. 
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ignored, nor is any priority attached to individual measures. State and local agencies are expected 
to achieve their negotiated performance targets within a 90 percent threshold across all measures, 
with sanctions kicking in when any one falls below agreed-upon levels.'® 


Finally, questions remain about the accuracy of accountability data because states only report 
raw data on the number of concentrators included in the numerator and denominator of each 
measure. States are neither required to detail the methodology used to collect information nor the 
manner in which state or local administrative records were accessed to report outcomes. For this 
and other reasons, NACTE is examining how state measures are constructed and data are 
collected. Case study visits to states and local agencies are providing rich, in-depth information 
on these topics. 


The organization of state data systems and state privacy laws also affects data quality and 
availability. Many states have been maintaining separate data systems for CTE concentrators 
that, in some cases, do not include individually identifiable data. When those data do exist, state 
CTE administrators in the past have had to conduct matches across one or more state databases, 
which can be challenging if a unique student identifier, such as a Social Security number, is not 
incorporated in the data file. The availability of funding from the American Recovery and 
Reinvestment Act of 2009 for developing and expanding state student-level databases promises to 
address many of these concerns and CTE administrators need to fully participate in designing 
and upgrading those databases. 


What are the additional accountability requirements if Tech Prep is continued? A state’s 
choice to continue separate funding for its Tech Prep program, rather than consolidating those 
funds into the basic grant program, requires additional accountability measures under Perkins IV. 
Specifically, states are required to maintain separate funding streams and to collect data on a set 
of nine Tech Prep performance indicators. These states also face the task of reporting on Tech 
Prep students who transition from the secondary to postsecondary levels, which has, in the past, 
proven to be a difficult undertaking. Because the U.S. Department of Education does not collect 
data on the measure used by each state as it is not required to negotiate performance levels for 
Tech Prep, the validity and reliability of Tech Prep data within and across states is as yet 
undetermined." 


How will performance negotiations affect accountability? Perkins IV extends the requirement 
that state agencies negotiate with the U.S. Department of Education a level of performance for 
each of the accountability measures contained within the legislation. In the case of the secondary 


'S Perkins IV puts forward a progressive set of sanctions that begin with a state agency or local provider being 
required to develop a program improvement plan to address areas in which performance data show a state or local 
entity has failed to meet at least 90 percent of an agreed-on performance level or target for any indicator. Monetary 
sanctions may be imposed if a state agency or local entity fails to implement an improvement plan, fails to make any 
improvement in meeting any of its levels of performance within the first program year of the implementation of its 
improvement plan, or fails to make improvement on its performance for the same core indicator for three 
consecutive years. 


'° The National Association for Tech Prep Leadership has convened a working group to develop measurement 
definitions including preferred constructions for the numerators and denominators of the Tech Prep indicators. To 
date, all states are agreeing to use these constructions, which will help to ensure the validity, reliability, and 
consistency of Tech Prep data. 
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measures, states have little latitude in the negotiation of some measures: states are expected to 

align their targets for their measures of secondary reading and mathematics skill attainment and 
secondary graduation rate with their state’s annual measurement objectives established through 
their ESEA negotiations.”° The negotiations for targets for all measures are on a two-year cycle. 


One substantial innovation in Perkins IV is the requirement that states reach agreement with local 
subgrantees on performance levels for each performance measure. In some states, all local 
entities receiving Perkins IV funds have agreed to accept the performance level negotiated by the 
state with the U.S. Department of Education; in others, locals negotiate with the state to set their 
own individual performance levels. In either instance, states must ensure that agreed-upon local 
performance levels will require that local entities make continual progress toward improving the 
performance of CTE students. 


Perkins IV further requires that, if a subgrantee does not meet at least 90 percent of its 
performance level on each core indicator every year, it must develop and implement an 
improvement plan to address its deficiencies. Gaining a clearer understanding of how the state 
and local negotiation process occurs, the outcomes of these discussions, and how local 
subgrantees are held accountable for their performance is a particular emphasis for the NACTE. 


2.2 Conclusion 


The NACTE studies are assessing state and local implementation of the Perkins IV legislation 
along with outcomes of CTE realized under earlier versions of the Perkins legislation. The final 
report will present these results. Study findings should provide useful information in assessing 
the effects of statutory changes introduced in Perkins IV, which include updated accountability 
indicators, the extension of performance accountability to the local subgrantee level, the 
introduction of CTE POS, and increased state flexibility in allocating Tech Prep funds. 


The remaining sections of the interim report will present initial findings with respect to 
secondary CTE including participation trends in CTE programs, student outcomes, and 
international benchmarking. 


°° States generally use their statewide ESEA Annual Measurement Objective (AMO) to establish their performance 
levels for the secondary academic attainment and their ESEA statewide targets for the high school graduation 
indicators although some have negotiated with the U.S. Department of Education to set different levels or targets 
under Perkins IV. As of May 2009 (excluding the District of Columbia and the territories), 48 states were using their 
statewide AMOs for reading and 49 states for math. A total of 37 states used their ESEA targets for high school 
graduation. 
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Chapter 3. 


Participation in Secondary Career and Technical Education from 
1982 to 2004 


Participation in career and technical education (CTE) at the secondary education level has 
changed significantly over the last three decades. This chapter~’ describes those changes based 
on national longitudinal studies of the educational experiences of the high school classes of 
1982, 1992, and 2004. These data provide a baseline against which to evaluate more recent 
student participation data that will be included in the final National Assessment of Career and 
Technical Education (NACTE) report. 


The background for the changes in CTE participation is the shift in which a majority of 
secondary graduates complete the academic curricular foundation defined by the new basics 
courses recommended in the 1983 report, A Nation at Risk (National Commission on Excellence 
in Education, 1983). These new basics are also touted more recently by ACT (2008b) as the 
foundation for a student being college and career ready.” The proportion of high school 
graduates choosing the academic foundation consisting of the new basics rose from 14 percent in 
1982 to 60 percent in 2004 (Dalton et al., 2013). 


A representative survey of employees covering white-collar, blue-collar, and service jobs 
indicate that employers need workers who have a foundation of both academic and technical 
skills but with differential skill levels required for different occupational categories (Exhibit 4). 
Jobs in all areas typically are filled with workers who have a high school diploma or more. In 
order to obtain white-collar and upper-blue-collar jobs, workers on average need more than a 
high school diploma and especially strong reading and writing skills. With respect to technical 
skills, white-collar jobs on average stress greater computer skills, and blue-collar jobs stress 
greater mechanical skills. Only service jobs on average have neither higher levels of computer or 
mechanical skill requirements, although this average may mask requirements in specific service 
sectors. 


The Carl D. Perkins Vocational and Applied Technology Education Act of 1990 (Perkins IT) 
supported students in CTE programs to take more rigorous academic courses along with 
technical courses. Succeeding Perkins legislation—including the Carl D. Perkins Career and 
Technical Education Act of 2006 (Perkins IV)—has also increasingly emphasized the importance 
of having students in CTE programs master rigorous academic and technical content. 


*! This chapter is based on From Track to Field: Trends in Career and Technical Education Across 3 Decades, by 
Ben Dalton and Erich Lauff, RTI International, and Robin Henke, Martha Alt, and Xiaojie Li, MPR Associates 
(Dalton et al., 2013). 


~° The core academic new basics consist of at least four Carnegie credits in English and three credits each in 
mathematics, science, and social studies. A Carnegie credit is equivalent to a course taken for one period each day 
for one full school year. 
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Exhibit 4. Percentage of U.S. workers ages 18-64 years old performing selected tasks on their 
job or requiring specified job skills or attributes, by occupational category: October 
2004—January 2006 


Service 

Task performed on Upper white Upper blue _Lower blue workers 
job or job skill collar (e.g., | Lower white collar (e.g., _ collar (e.g., (e.g., food, 
needed (See All management, — collar(e.g., craft repair, factory work, home health, 
footnotes for score workers’ professional, clerical and construction, truck child care, 
codes) (average) technical) sales) mechanics) driving) police) 
Education required 

(3 = HS grad.; 

6 = BA)® 4.15 5.42 3.72 3.49 3.10 3.17 
Complexity of 

computer skills 

used? 4.21 5.91 5.06 1.95 2.43 177 
Math skills—% using 

any math 94 95 97 94 91 88 
Algebra | (%) 19 30 8 36 16 4 
Reading—any 

reading (%) 96 99 97 91 91 95 
Documents 5 or more 

pages (%) 54 81 47 46 26 32 
Writing—do writing 

(%) 91 99 93 83 80 83 
Use forms (%) 67 78 77 61 46 46 
Mechanical skill level® 2.50 1.73 1.38 5.97 4.55 2.12 


Exhibit reads: The average education level required by all U.S. workers is 4.15 (above high school). The percentage 
of all U.S. workers using math in their jobs is 94. 

* 1 = eighth grade or less; 2 = high school dropout; 3 = high school graduate; 4 = high school + vocational education; 5 = some 
college or two-year degree; 6 = bachelor’s degree; 7 = more than bachelor’s degree. 

> Self-rated complexity of computer skills used on job (0 = no computer use, 1 = very basic, 10 = very complex). 

° Self-rated complexity of mechanical skills used on job (0 = very basic, 10 = very complex). 

NOTE: The Survey of Skills, Technology, and Management Practices (STAMP) is a random-digit-dial telephone survey of employed 
wage and salary workers in the United States at least 18 years of age conducted between October 2004 and January 2006 

(n = 2,304). Eligible individuals were selected randomly within households. All respondents were interviewed with respect to their 
own jobs. 

SOURCE: Michael J. Handel, A New Survey of Workplace Skills, Technology, and Management Practices (STAMP): Background 
and Descriptive Statistics, May 23, 2007; NAS presentation. 

http://www7.nationalacademies.org/cfe/Future_Skill_ Demands_Michael_Handel_Paper.pdf. 


In tracing the change in CTE participation between 1982 and 2004 as school systems strengthen 
enrollment in the new basics, this chapter follows the National Assessment of Vocational 
Education (NAVE; Silverberg et al., 2004) in focusing on CTE investors. CTE investors include 
two categories of students— 


* CTE concentrators who earn three or more credits in one CTE occupational area, and 
* CTE explorers who earn three or more credits in any CTE occupational areas. 


This chapter starts by describing data sources for the analyses of CTE participation trends. It next 
examines trends in CTE investor participation within the context of overall trends in academic 
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and enrichment coursetaking. It then compares CTE investors and non-investors with respect to 
academic and employment experiences including (1) coursetaking in math and science, 

(2) mathematics achievement, (3) postsecondary expectations and initial postsecondary 
enrollment, and (4) post-high school employment experiences within two years of graduation. 


3.1. Data Sources for the Analyses 


Data used in this chapter come from three national longitudinal studies tracking the secondary 
and postsecondary experiences of the high school graduating classes of 1982, 1992, and 2004. 
The three studies are the High School and Beyond Longitudinal Study (HS&B) for the high 
school class of 1982; the National Education Longitudinal Study of 1988 (NELS:88) for the class 
of 1992; and the Education Longitudinal Study of 2002 (ELS:2002) for the class of 2004. While 
these data do not cover the period since Perkins IV was enacted, they are the most recent 
longitudinal data on high school cohorts from the National Center for Education Statistics 
(NCES) with the necessary follow-up period required for the analysis.”* 


These three studies provide information on the depth and extent of CTE participation, 
independently and in relation to: academic and other subjects; the social and demographic 
characteristics of graduates; their high school experiences and achievement outcomes; and their 
initial postsecondary education and employment experiences. A particular advantage of the 
longitudinal studies is their collection of achievement outcomes and post-high school academic 
and employment experiences. All studies were conducted by NCES, which is part of the Institute 
of Education Sciences in the U.S. Department of Education. 


All three studies share similar content and design enabling cross-study comparability over time. 
All include a nationally representative sample of seniors. As part of the senior-year surveys, each 
study gathered and systematically coded high school transcripts. All comparisons between values 
discussed in this chapter’s text (changes over time, differences between groups, and differences 
between groups in changes over time) are statistically significant at p < .05, unless explicitly 
noted as nonsignificant. The tests for statistical significance are based on point estimates and 
standard errors that account for the complex nature of the studies’ samples. The samples used 
were restricted to public high school graduates who had a complete set of transcripts. 


3.2 Measuring CTE Participation 


This analysis focuses on student enrollment in CTE occupational courses offering specific labor 
market preparation. CTE courses in general fall into three groups: 


¢ Family and consumer sciences education (FCSE) 
* General labor market preparation (GLMP) 


¢ Specific labor market preparation (SLMP) or occupational education 


*? The final report of the NACTE will incorporate data from the 2009 High School Transcript Study of NCES that 
will have Perkins IV CTE participation data. 


National Assessment of Career and Technical Education: Interim Report 21 


FCSE courses prepare students for family and consumer roles outside the paid labor market. 
GLMP courses teach general employment skills that are not specific to one occupational area, 
such as basic computer literacy, keyboarding or typing and general work experience courses. In 
contrast, occupational courses (SLMP education) are designed to prepare students for work in a 
specific occupational field or for related postsecondary education or training. 


The CTE occupational courses being studied are divided into the following 11 mutually 
exclusive occupational areas (Bradby and Hudson, 2008; Hudson and Laird, 2009): 

¢ Agriculture and Natural Resources; 

¢ Architecture, Construction, and Science Technology; 

* Business; 

¢* Communications and Design; 

¢ Computer and Information Science; 

¢ Consumer and Culinary Services; 

¢ Engineering Technologies; 

* Health Sciences; 

¢ Manufacturing, Repair, and Transportation; 

° Marketing; and 

¢ Public Services. 
Measuring participation in CTE in terms of occupational coursetaking can be done in two ways: 
the number of CTE occupational credits earned or some threshold number of courses 
representing a significant investment in CTE (e.g., Hoachlander, Kaufman, and Levesque, 1992). 
Continuous measures of the number of credits may be less useful because their representation as 
numerical averages can mask substantial variation within the population and overstate the 
influence of particularly high or low values. In considering whether CTE has particular outcomes 
associated with it, some minimal level of exposure seems necessary. In particular, some level of 


concentration in CTE occupational courses may be particularly important for postsecondary 
work opportunities. 


This descriptive report expresses a concentration level for CTE participation in terms of CTE 
investors (Silverberg et al., 2004). This category is defined on the basis of the number of 
occupational courses a student takes. Those taking three or more are classified as CTE investors, 
while those taking fewer than three are non-investors. 


CTE investors are also divided into two groups of students (Silverberg et al., 2004). Those who 
earn their three or more occupational credits in one occupational area are CTE concentrators. In 
contrast, those who earn their three or more occupational credits in more than one area are called 
CTE explorers (see Exhibit 5). 
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Exhibit 5. | Types of CTE investors 


CTE investor: Earns three or more CTE occupational credits 


CTE concentrator: Earns three or more occupational credits in one area (but may earn more credits in 
other occupational areas) 


CTE explorer: Earns three or more occupational credits, but fewer than three credits in a single 
occupational area 


NOTE: CTE = career and technical education. 


These categories build upon the work of a number of previous analyses (Arum and Shavit, 1995; 
Hudson and Laird, 2009; Levesque, 2003a, 2003b; Levesque et al., 1995, 2000, 2008; Silverberg 
et al., 2004). Their advantage is twofold. First, although the categories are discrete, they reflect 
the fact that participation in CTE occupational coursetaking is a matter of degree. Almost all 
students (96 percent) took at least one occupational course in 2004. But most states do not report 
students in their Perkins IV secondary level performance data unless they are a CTE 
concentrator, which they define in terms of a threshold level of participation in occupational 
courses, usually three courses in one occupational area, but sometimes two when only two are 
offered in an area. 


But as the data below show, students today are increasingly likely to take three or more CTE 
courses in more than one occupational area. The pattern of CTE coursetaking could reflect the 
common perception that most new entrants to the labor force should expect to change their 
careers several times over the course of their lives, and investing in CTE courses in high school 
is one way to become familiar with possibly appealing alternative careers. And that is the second 
advantage of this typology, namely, that it recognizes students take different pathways toward 
investing in CTE, even if the credits are in more than one occupational area. 


In measuring students’ participation in CTE occupational coursetaking, high school transcripts 
for each graduating class were the primary data source. Participation in specific CTE programs, 
enrollment in certain types of high schools, and coursetaking experiences could all be used, 
independently or jointly, to define CTE students. However, coursetaking data from transcripts 
provide the most precise information about student participation in CTE. The specific course 
credits found on transcripts enable the construction of the categories of CTE investment.” 


** Tn this report, the terms CTE investor, CTE concentrator, and CTE explorer all use the same definitions employed 
in the NAVE (Silverberg et al., 2004). However, the classification of occupational courses has changed since NAVE 
was issued so that the numbers reported here cannot be directly compared to that report. Since the NAVE was 
released, NCES revised the CTE portion of its Secondary Schools Taxonomy of courses used for transcript analyses 
and made two consequential changes. First, a total of 21 courses were moved from occupational areas into FCSE or 
GLMP categories. Most importantly, all home economics, introduction to computers, and keyboarding courses were 
removed from occupational area categories. Second, a large occupational area used in the NAVE report, trade and 
industry, was partitioned into separate occupational areas or combined with smaller existing occupational areas. 
Both of these changes mean that smaller proportions of students are now identified as occupational concentrators 
than in the NAVE. This report also follows the conventions of NCES in defining occupational courses. 
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3.3. Participation in CTE 


Participation in career and technical education in high school occurs in the context of meeting 
curriculum requirements for high school graduation. Over the past quarter century or so, those 
requirements have increased, especially the academic credits needed to earn a diploma. Those 
increasing requirements are part of a longstanding high school reform movement going back at 
least to the early 1980s (e.g., National Commission on Excellence in Education, 1983) that 
started because of concern about the level of academic achievement of high school graduates. 


Graduates Earning an Increasing Number of Total Course Credits 


One of the most striking findings regarding coursetaking patterns of high school graduates since 
1982 is the large increase in the number of credits (Carnegie units) earned (reminder: all changes 
or differences reported in the text are statistically significant unless otherwise noted). The total 
number of credits earned by high school graduates went from 21.7 credits in 1982 to 26.2 credits 
in 2004, an increase of over one-fifth (Exhibit 6 and Exhibit 7). The increasing credits earned 
were in academic and enrichment fields.”> Academic credits earned increased from 14.6 credits 
to 19.2 credits and enrichment credits from 2.6 credits to 3.2 credits between 1982 and 2004. 
More recent graduates increased their academic coursetaking in every field between 1982 and 
2004 (Exhibit 7), with the largest increases in science (from 2.3 credits to 3.3 credits) and in non- 
English language (from 1.0 credits to 2.0 credits). Over the same 1982-2004 period, CTE 
coursetaking diminished from 4.6 credits to 3.8 credits, with the decrease concentrated between 
1982 to 1992. 


CTE Credits Are a Smaller Share 


A major issue for many CTE educators has been that increasing academic course requirements 
could squeeze out participation in CTE courses, especially occupational courses. In looking at 
the period between 1982 and 2004, the share of CTE credits earned by high school graduates 
dropped from over a fifth of all credits in 1982 to about 14 percent in 2004 (Exhibit 8). Students 
continued, however, to take a significant number of CTE courses. While total CTE credits earned 
dropped between 1982 and 2004, in the last year students still earned, on average, 3.8 CTE 
credits. 


°° Enrichment credits include courses in physical education, religion, and military education. 
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Exhibit 6. Average number of secondary credits earned by public high school graduates, by 
type of course work: 1982, 1992, and 2004 
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Exhibit reads: In 1982, the average number of academic credits earned by public high school graduates was 

14.6 credits. 

NOTE: CTE = career and technical education. See Exhibit 7 for details on data. Enrichment credits include courses such as physical 
education, religion, and military education. A credit (Carnegie unit) is equivalent to a course taken for one period each day for one 
full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 7. Average number of credits (Carnegie units) earned by public high school graduates, 
by subject area: 1982, 1992, and 2004 


Total and subject area 1982 1992 2004 
Total credits earned 21.7 24.0 26.2 
Total career and technical education (CTE) credits earned 4.6 3.9 3.8 
Family and consumer sciences education 0.5 0.4 0.3 
General labor market preparation 1.0 1.0 0.9 
Occupational area credits, total 3.0 2.4 2.6 
Agriculture and natural resources 0.2 0.2 0.2 
Architecture, construction, and science technology 0.1 0.1 0.1 
Business 1.0 0.7 0.5 
Communications and design 0.2 0.2 0.3 
Computer and information science 0.1 0.2 0.4 
Consumer and culinary services 0.2 0.2 0.2 
Engineering technologies 0.2 0.2 0.2 
Health sciences 0.1 0.1 0.1 
Manufacturing, repair, and transportation 0.7 0.4 0.3 
Marketing 0.1 0.1 0.1 
Public services 0.1 # 0.1 
Total academic credits earned 14.6 17.4 19.2 
English 4.0 4.2 4.3 
Mathematics 2.7 3.3 3.6 
Science 2.3 3.0 3.3 
Social studies 3.2 3.5 3.9 
Fine arts 1.5 1.7 2.1 
Non-English language 1.0 1.8 2.0 
Total enrichment credits earned* 2.6 2.7 3.2 


Exhibit reads: In 1982, the average total credits earned by public high school graduates was 21.7 credits. 

# Rounds to zero. 

* Enrichment credits include courses such as physical education, religion, and military education. 

NOTE: A Carnegie unit (credit) is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 8. Percentage of total credits earned by public high school graduates, by type of 
course work: 1982, 1992, and 2004 
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Exhibit reads: For public high school graduates, the percentage of total credits that were academic was 67 in 1982. 


NOTE: CTE = career and technical occupation. Sums may not total to 100 percent because of rounding. Enrichment credits include 
courses such as physical education, religion, and military education. A credit (Carnegie unit) is equivalent to a course taken for one 
period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


CTE Participation Patterns Are Also Changing 


Patterns of CTE coursetaking also changed over this period as public high school graduation 
requirements rose (Exhibit 9). The share of graduates who are CTE investors—those who earn 
three or more occupational credits—declined from 46 percent in 1982 to 35 percent in 1992, but 
that share has since stabilized and increased between 1992 and 2004 from 35 percent to 

38 percent of high school graduates. The proportion of those earning three or more credits in one 
occupational area, the CTE concentrators, declined from 30 percent in 1982 to 17 percent in 
2004, but the share of CTE explorers who earned three or more CTE occupational credits in 
more than one occupational area increased from 16 percent in 1982 to 21 percent in 2004. 
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Exhibit 9. Percentage of high school graduates participating in CTE, by level of CTE 
occupational coursetaking: 1982, 1992, 2004 
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Exhibit reads: The percentage of public high school graduates with any CTE credit was 98 percent in 1982. 


NOTE: CTE = career and technical education, CTE concentrators earned three or more credits in at least one occupational area. 
CTE investors consist of CTE concentrators plus CTE explorers who earn three or more total occupational credits but did not 
concentrate in an occupational area. A Carnegie unit is equivalent to a course taken for one period each day for one full school year. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


This shift meant that investing in CTE went from being predominantly CTE concentrators to a 
majority of investors being CTE explorers (Exhibit 10). Almost two-thirds of students investing 
in CTE were CTE concentrators in 1982, but about 55 percent were CTE explorers by 2004. 
Note also that the share of graduates earning at least some CTE occupational credit declined only 
slightly over this period, from 98.0 percent in 1982 to 97.6 percent in 1992 to 95.6 percent in 
2004, and still included virtually all students. 


This shift in the composition of investors away from concentrators to explorers has implications 
for CTE performance reporting at the secondary level. As noted in Chapter 2, federal guidance 
suggests that states report the performance only for secondary CTE concentrators, who currently 
account for less than half (45 percent) of all CTE students who take three or more CTE courses 
(investors). Hence, no state performance information is available on the majority (55 percent) of 
CTE investors who take more than three courses but in two or more occupational areas. 
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Exhibit 10. Percentage of CTE investors, by level of CTE course concentration: 1982, 1992, and 
2004 
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Exhibit reads: In 1982, the percentage of CTE explorers was 35 percent. 


NOTE: CTE = career and technical education. CTE concentrators earned three or more credits in at least one occupational area. 
CTE explorers earned three or more total occupational credits but did not concentrate in an occupational area. A credit (Carnegie 
unit) is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


3.4 CTE Investors’ Characteristics 


Concern about the student characteristics of those earning high school credits in CTE 
occupational courses goes back to when CTE was called vocational education and seen as an 
educational track for low-performing students. In some cases, students were counseled (or even 
directed) into occupational courses based on their personal characteristics, not their preferences, 
aspirations, and abilities. This section examines how patterns of participation by gender, race or 
ethnicity, and socioeconomic status have changed between 1982 and 2004 among CTE investors 
(concentrators and explorers combined). 


Gender 


Between 1982 and 2004, patterns of participation in career and technical education by gender 
became less similar (Exhibit 11). In 1982, about 49 percent of males and 43 percent of females 
were CTE investors. In contrast, by 2004, about 45 percent of males were CTE investors, but 
only about 32 percent of females were CTE investors. Within the category of CTE investor, in 


National Assessment of Career and Technical Education: Interim Report 29 


1982 females and males were approximately equally likely to be CTE concentrators (Exhibit 24). 
By 2004, males (21 percent) were more likely than females (14 percent) to be CTE 
concentrators. 


Exhibit 11. Percentage of public high school graduates who are CTE investors, by gender: 1982, 
1992, and 2004 
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Exhibit reads: In 1982, the percentage of male public high school graduates who were CTE investors was 
49 percent. 
NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). A credit (Carnegie unit) is equivalent to a course taken for one period each day for one full 
school year. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


Race and Ethnicity 


Between 1982 and 2004, patterns of participation in career and technical education by race or 
ethnicity became more similar (Exhibit 12 and Exhibit 24). The general pattern was for the 
proportions of CTE investors, particularly CTE concentrators, to decline among all racial and 
ethnic groups except Asian or Pacific Islanders over that period. Hispanics had a larger decrease 
than blacks or whites, with Hispanic investors declining by 20 percentage points, compared with 
9 and 4 percentage point declines for black and white investors, respectively. Within CTE 
investors, by 2004 blacks and whites were most likely to be CTE concentrators (about 18 percent 
and 19 percent, respectively), followed by Hispanics (about 13 percent) and Asian or Pacific 
Islanders (about 10 percent). 
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Exhibit 12. Percentage of public high school graduates who are CTE investors, by selected race 
or ethnicity: 1982, 1992, and 2004 
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Exhibit reads: In 1982, 28 percent of Asian or Pacific Islander public high school graduates were CTE investors. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). A credit (Carnegie unit) is equivalent to a course taken for one period each day for one full 
school year. The category Asian or Pacific Islander includes Native Hawaiians. Hispanic may be of any race. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


Socioeconomic Status 


Patterns of participation in CTE also became more similar over time when graduates who are 
CTE investors are grouped by socioeconomic status (SES) quartiles. The major shift was the 
decline in the share of lower SES quartile graduates who were CTE investors from about 

58 percent to about 44 percent and of middle SES quartiles graduates from 48 percent to 

41 percent (Exhibit 13). The share of upper SES quartile graduates who were CTE investors was 
basically unchanged at about 29 percent between 1982 and 2004. Within the category of CTE 
investors, CTE concentrators declined from about 40 percent to about 21 percent among lower 
SES quartiles, while the share of upper SES quartile graduates who were CTE concentrators saw 
no decline (Exhibit 24). 


Although graduates grouped by SES quartile converged in their patterns of CTE participation 
between 1982 and 2004, important differences among the groups remained. Among graduates in 
the lower SES quartile about 21 percent were CTE concentrators in 2004, while only about 

12 percent in the upper SES quartile were (Exhibit 24). 
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Exhibit 13. Percentage of public high school graduates who are CTE investors, by 
socioeconomic status quartile: 1982, 1992, and 2004 
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Exhibit reads: In 1982, 58 percent of the lowest socioeconomic status—quartile public high school graduates were 
CTE investors. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). A credit (Carnegie unit) is equivalent to a course taken for one period each day for one full 
school year. Socioeconomic status is based on parental income, education, and occupation. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


3.5 CTE Investors’ Academic and CTE Occupational Coursetaking 


After looking at new basics coursetaking by CTE investors and non-investors, this section 
examines how coursetaking in mathematics and science among investors and non-investors has 
changed between 1982 and 2004. 


New Basics Coursetaking 


The share of students completing the core academic new basics courses recommended in the A 
Nation at Risk report (National Commission on Excellence in Education, 1983) and by ACT 
(2008b) increased for both CTE investors and non-investors between 1982 and 2004. Among all 
graduates, the percent completing the new basics rose from 14 percent in 1982 to 47 percent in 
1992 and 60 percent in 2004. Among CTE investors, only about 6 percent completed the new 
basics in 1982, compared with about 22 percent of non-investors (Exhibit 14). By 2004, about 
58 percent of CTE investors had completed the new basics, compared with about 61 percent of 
other students, a gap that is not statistically significant. Hence, there is no longer a discernable 
gap between CTE investors and other students in completing the new basics as preparation for 
college and career. CTE investors have substantially closed the academic gap with non-investors 
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due in part by increasing academic course requirements as a condition for graduation in most 
states. Additionally, a small set of new courses that generally enrolled higher achieving students 
(e.g, in computer science and education) became classified as CTE. 


Exhibit 14. Percentage of CTE investors and non-investors taking the new basics courses: 1982, 
1992, 2004 
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Exhibit reads: In 1982, 6 percent of CTE investors completed the new basics courses. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors earned zero to less than three total occupational credits (Carnegie units). A 
credit (Carnegie unit) is equivalent to a course taken for one period each day for one full school year. The new basics courses 
include four credits of English, three credits of math, and two credits each of science and social studies. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


Highest Mathematics and Science Courses Taken 


The percentages of CTE investors and other high school graduates taking various levels of math 
and science courses are shown in Exhibit 15 and Exhibit 16 (see also Exhibit 25 and Exhibit 26). 
For mathematics, courses were divided into five levels: level 1 = no math credits or low- 
academic math only; level 2 = algebra I or plane geometry; level 3 = algebra II; level 4 = algebra 
IH, trigonometry, or analytic geometry; and level 5 = precalculus or calculus (Burkam and Lee, 
2003). If a high school graduate earned at least one credit at one of the five levels, in any grade, 
he or she was counted in that level. For science, courses were divided into four levels: level 

1 = no science credits or low-level science only; level 2 = secondary physical science (e.g., earth 
science) or basic biology; level 3 = general biology; or level 4 = advanced biology, chemistry, or 
physics. Again, if a graduate earned at least one credit at a given level, in any grade, he or she 
was counted in that level. Note that the composition of the CTE investor and non-investor group 
changed between 1982 and 2004 as the percentage of investor students in the lowest 
socioeconomic group declined. 
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Exhibit 15. Percentage of public high school graduates who were CTE investors and non- 
investors taking advanced mathematics (above geometry and algebra Il): 1982, 1992, 
and 2004 
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Exhibit reads: In 1982, 12 percent of CTE investors took advanced mathematics. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors earned zero to less than three total occupational credits (Carnegie units). A 
credit (Carnegie unit) is equivalent to a course taken for one period each day for one full school year. Advanced math includes at 
least one course in algebra Ill, trigonometry, analytic geometry, precalculus, calculus, and any Advanced Placement or international 
baccalaureate course. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 16. Percentage of public high school graduates who were CTE investors and non- 
investors taking advanced science courses (advanced biology, chemistry, or 
physics): 1982, 1992, and 2004 
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Exhibit reads: In 1982, 20 percent of CTE investors took advanced science courses. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors earned zero to less than three total occupational credits (Carnegie units). A 
credit (Carnegie unit) is equivalent to a course taken for one period each day for one full school year. Advanced science includes at 
least one course in chemistry II, physics II, or an advanced biology course such as anatomy, cell biology, or genetics. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


The general pattern is that credits earned in the highest-level math and science courses increased 
for CTE investors as well as for non-investors between 1982 and 2004, and the gap between 
CTE investors and non-investors has narrowed. For example, in 2004, in mathematics about 

39 percent of CTE investors earned math credits at the level of algebra III, precalculus, or 
calculus courses, up from 12 percent in 1982. While non-investors also increasingly earned 
higher-level math credits, the gap in advanced math credits between CTE investors and non- 
investors decreased by 9 percentage points from 24 percentage points to 15 percentage points 
between 1982 and 2004. 


CTE investors also earned more credits in advanced science courses and decreased the advanced 
science gap, compared with non-investors between 1982 and 2004 (Exhibit 15 and Exhibit 26). 
In 1982, only 20 percent of CTE investors earned credits in advanced biology, chemistry, or 
physics; by 2004, 60 percent of CTE investors did. Non-investors also were more likely to earn 
advanced credits in science over the period from 1982 to 2004, but the gap in credits earned in 
advanced science courses between CTE investors and non-investors decreased from 

28 percentage points to 13 percentage points. 
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3.6 Math Achievement 


There is considerable interest in the academic achievement of secondary CTE students. Perkins 
IV requires states to report on the academic achievement of CTE concentrators using the 
measures of math and reading or English language arts achievement required by the Elementary 
and Secondary Education Act of 1965 as amended. Previous legislation had also called for a core 
indicator that measured student attainment of academic (as well as vocational and technical skill) 
proficiencies. 


This section looks at how the academic achievement of CTE investors in math changed over 
time, both independently and relative to non-investors’ achievement, and focuses on math 
achievement and its relationship to CTE coursetaking in the graduating classes of 1992 and 
2004. Math achievement is based on a standardized math test developed for the NCES 
longitudinal studies, and students were high school seniors when they took the test. While 
mathematics knowledge and skills were also assessed for graduates in 1982, only the 1992 and 
2004 math tests are on the same psychometrically developed scale (O—81) that permits 
comparisons of change over time. The math test is the only assessment comparable across the 
longitudinal studies being used. Note also that the composition of the CTE investor and non- 
investor group changed, with the relative percentage of investor students in the lowest 
socioeconomic group declining. 


Exhibit 17 shows the overall mathematics number-right scores for high school graduates in 1992 
and 2004 for CTE investors and non-investors. While math scores for non-investors showed no 
change over time, the overall score of CTE investors increased from 46 correct items in 1992 to 
48 correct items in 2004. However, in both 1992 and 2004, CTE investors scored lower in math 
achievement than non-investors. 
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Exhibit 17. Average estimated number-right math scores of CTE investors and non-investors: 
1992 and 2004 
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Exhibit reads: In 1992, CTE investors had an average math score of 46. 


* Estimated number-right math scores represent the item response theory—estimated number correct students would have answered 
had they received all test questions in the NELS:88 test item pool; the scale runs from 0-81 and has been equated across the two 
years. 

NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors earned zero to less than three total occupational credits (Carnegie units). A 
credit (Carnegie unit) is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


3.7 Postsecondary Education 


Economic competition in the global economy makes it increasingly important that high school 
graduates gain higher-level skills and knowledge only obtainable from postsecondary education 
or training. Even if graduates focus on occupational areas while in high school and do not plan to 
obtain a baccalaureate degree, most white-collar positions in many fields require further training 
and more advanced skills than can be learned in high school (Exhibit 4). Examining the 
expectations and initial postsecondary education experiences of high school graduates can help 
identify the areas of success and weakness in CTE investors’ high school training. Each of the 
three longitudinal surveys used for analyses in this report include data on public high school 
graduates’ educational expectations in the 12th grade, as well as information on postsecondary 
educational status in the two years immediately following graduation. 
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Senior-Year Educational Expectations 


Educational expectations rose for both groups of graduates—CTE investors and non-investors— 
between 1982 and 2004 (Exhibit 18 and Exhibit 27). But the educational expectations of CTE 
investors rose more than non-investors. Between 1982 and 2004, the proportion of CTE investors 
who expected to get a bachelor’s degree or higher increased from about 26 percent to about 

60 percent, while among non-investors those expectations increased from about 58 percent to 
about 76 percent. As a result, the gap in educational expectations between CTE investors and 
non-investors was reduced by 15 percentage points (value differs from exhibit due to rounding), 
although CTE investors still have lower educational expectations than non-investors. 


Exhibit 18. Percentage of public high school graduates expecting to earn a bachelor’s degree or 
more, by CTE occupational investment: 1982, 1992, and 2004 
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Exhibit reads: In 1982, 26 percent of CTE investors expected to earn a bachelor’s degree or more. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors earned zero to less than three total occupational credits (Carnegie units). A 
Carnegie unit is equivalent to a course taken for one period each day for one full school year. “Bachelor's degree or more” refers to 
expectations of a four-year college degree, a master’s or equivalent degree, or an advanced graduate or professional degree. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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A large change in educational expectations occurred at the level of completing no more than high 
school (Exhibit 19 and Exhibit 27). The proportions of both CTE investor and non-investor 
students expecting to complete no more than a high school education dropped over time, but the 
decline was greater among CTE investors. Fewer than 10 percent of high school graduates who 
are CTE investors now expect to complete no more than a high school education, compared with 
one-quarter in 1982. These graduates seem to have received the message that a high school 
diploma is no longer enough for getting a good job in today’s more competitive global economy. 


Exhibit 19. Percentage of public school graduates expecting to earn no more than a high school 
degree, by CTE occupational investment: 1982, 1992, and 2004 
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Exhibit reads: In 1982, 25 percent of CTE investors expected to earn no more than a high school degree. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). Non-investors earned zero to less than three total occupational credits (Carnegie units). A 
Carnegie unit is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


Ever Attended Postsecondary School 


Between 1982 and 2004, high school graduates who were CTE investors increased their 
postsecondary attendance during their first two years after high school (Exhibit 20). Among CTE 
investors, that proportion increased from 54 percent in 1982 to 75 percent in 2004. Among non- 
investors, that proportion also increased, but not as rapidly, with the result being that the gap 
between CTE investors and non-investors in enrolling in postsecondary education in the first two 
years after high school graduation decreased by 15 percentage points, from 23 percentage points 
to 8 percentage points (value differs from exhibit due to rounding). 
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Exhibit 20. Percentage of public high school graduates ever enrolled in a postsecondary 
education institution in the first two years after graduation, by CTE occupational 
investment: 1984, 1994, and 2006 
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Exhibit reads: In 1984, 54 percent of high school CTE investors had ever enrolled in a postsecondary education 
institution. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (regardless of 
concentration in any occupational area). A Carnegie unit is equivalent to a course taken for one period each day for one full school 
year. Non-investors earned zero to less than three total occupational credits (Carnegie units). “Postsecondary education institution” 
refers to both two-year and four-year institutions as well as private and public schools. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/84), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992” and “Third Follow-up, 1994”; and Education Longitudinal Study of 2002 (ELS:2002), 
“First Follow-up, High School Transcript Study, 2004” and “Second Follow-up, 2006.” 


3.8 Post—High School Employment 


Because one of the goals of CTE is to prepare students for work as well as further education after 
high school, the occupational expectations and initial post-high school work experiences are 
important outcomes to consider in evaluating trends in outcomes associated with CTE 
occupational coursetaking. Because the high school graduates of 2004 have been surveyed only 
once after graduation, the only comparable information across the three cohorts studied here is 
restricted to the initial two years after graduation. Given that entering the labor force can be a 
multiyear transition for many high school graduates, even for those who do not attend college, 
these initial experiences represent only part of the association between occupational coursetaking 
and work outcomes. 
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High School Senior- Year Occupational Expectations at Age 30 


Graduates who were CTE investors have both similarities and differences with non-investors 
regarding their occupational expectations at age 30 (Exhibit 21). The most common occupational 
expectation at that age in all three cohorts among CTE investors and non-investors alike is a 
professional job. What is especially striking about the expectation is its increase among CTE 
investors, rising from about 27 percent of such graduates expecting to be in a professional 
occupation by age 30 in 1982 to almost 54 percent in 2004. By 2004, the proportion of non- 
investors expecting a professional occupation by age 30 had increased to almost 70 percent, but 
the gap between non-investors and CTE investors had decreased from 25 percentage points to 16 
percentage points. 


Exhibit 21. Percentage of public high school graduates with given expectations of occupation at 
age 30, by CTE occupational investment: 1982, 1992, and 2004 


CTE CTE CTE Non- Non- Non- 
Occupational investor, investor, investor, investor, investor, investor, 
expectation 1982 1992 2004 1982 1992 2004 
Clerical 13.5 5.5 0.7 4.6 2.1 0.5 
Craftsperson 10.4 5.7 8.1 3.4 0.7 2.7 
Farmer 3.4 2.0 0.6 1.0 0.4 a 
Homemaker 2.6 0.8 0.2 2.2 0.9 t 
Laborer 1.2 1.1 1.0 0.8 0.1 0.3 
Manager 7.5 5.5 4.9 9.0 3.7 3.9 
Military 2.4 2.9 1.6 1.8 1.9 1.5 
Operative 4.2 2.2 1.6 1.7 1.4 0.1 
Professional 27.4 42.0 53.7 51.9 59.8 69.5 
Proprietor 5.0 6.3 4.7 3.4 4.9 3.2 
Protective services 1.7 4.3 3.5 1.5 3.6 4.2 
Sales 1.7 1.3 1.8 2.0 2.1 1.9 
Service 4.4 3.6 7.0 2.9 2.3 6.1 
Technical 11.5 7.7 9.7 11.2 5.6 5.3 
Other 3.5 9.2 1.0 2.7 10.6 0.6 


Exhibit reads: The percentage of CTE investors who expected to have a clerical occupation at age 30 was 
13.5 percent in 1982. 


+ Reporting standards not met. 

NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (Carnegie units). Non- 
investors earned less than three total occupational credits. A Carnegie unit is equivalent to a course taken for one period each day 
for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 


Another notable change among CTE investors includes a large drop in the proportion of 
graduates expecting to have a clerical occupation by age 30—no doubt, a tribute to technological 
change in the form of computers and word processing. Over the same period, expectations for 
being in the occupations of craftsperson and technical worker remained relatively popular for 
CTE investors (especially CTE concentrators) across all three cohorts. For example, about 

18 percent of CTE investors expected one of those jobs in 2004, about the same percentage as in 
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1982. Another interesting finding is that few CTE investors and non-investors in 2004 had 
expectations for being a manager. 


First Job Type of Non-College Attendees 


High school graduates in 1982, 1992, and 2004 who had no postsecondary attendance in their 
first two years after high school answered a series of questions two years after graduation 
regarding their first post-high school job. Distributions of these first-job types are shown in 
Exhibit 22. 


Exhibit 22. Percentage of non-college-attending public high school graduates in first job type, 
by CTE occupational investment: 1984, 1994, and 2006 


CTE CTE CTE Non- Non- Non- 

investor, investor, investor, investor, investor, investor, 

Type of first job 1984 1994 2006 1984 1994 2006 
Clerical 18.5 20.0 13.5 18.0 25.5 13.3 
Craftsperson 10.5 9.3 11.4 6.5 3.3 6.9 
Laborer or farmer 15.8 21.4 13.2 14.0 21.3 8.4 
Skilled operative 11.8 8.2 11.1 11.5 5.4 6.8 
Sales or service 33.1 26.3 39.4 37.2 32.9 50.4 
Managerial 2.9 6.0 5.4 3.3 5.2 6.4 
Other 7.4 8.9 6.0 9.5 6.5 7.7 


Exhibit reads: The percentage of high school CTE investors who had a clerical first job was 18.5 percent in 1984. 


NOTE: CTE = career and technical education. CTE investors earned three or more total occupational credits (Carnegie units). Non- 
investors earned less than three total occupational credits. A Carnegie unit is equivalent to a course taken for one period each day 
for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/84), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992” and “Third Follow-up, 1994”; and Education Longitudinal Study of 2002 (ELS:2002), 
“First Follow-up, High School Transcript Study, 2004” and “Second Follow-up, 2006.” 


Sales or service occupations were the most commonly reported first-job type among all 
coursetaking groups in 1984 and 2006 (which were two years after high school graduation in 
1982 and 2004, respectively). About 50 percent of non-investors reported holding that kind of 
first-job, while about 40 percent of CTE investors did. Among non-investors, the second most 
common first job was clerical, reported by about 13 percent, which was similar to the 14 percent 
of CTE investors reporting that kind of job. 


But among CTE investors, about 22 percent reported holding a first job as a craftsman or skilled 
operative, compared with about 14 percent of non-investors. That experience appears to be 
consonant with the expectations and technical training of many CTE investors noted above. CTE 
investors were also more likely to report being a laborer or farmer in their first job (about 

13 percent) than were non-investors (about 8 percent). Similar and small proportions (about 

5 percent or 6 percent) of both groups reported a first job as a manager. 


3.9 Conclusion 
Participation in CTE in high school changed significantly over the 22-year period between 1982 


and 2004—from before the Carl D. Perkins Act of 1984 (Perkins I) to after the Carl D. Perkins 
Vocational and Technical Education Act of 1998 (Perkins III). The backdrop for the changes in 
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CTE participation was the overall shift from 14 percent to 60 percent between 1982 and 2004 of 
high school graduates completing a new basics academic foundation (earning at least four credits 
in English, and three credits each in mathematics, science, and social studies). A Nation at Risk 
recommended these new basics in 1983, and from 1990 on, Perkins legislation authorizing 
funding for CTE supported the new basics courses. 


As the shift to more academic coursetaking occurred, participation in CTE changed in relation to 
total courses. Although the average number of CTE credits earned dropped between 1982 and 
1992 (from 3.0 credits to 2.4 credits), it increased somewhat between 1992 and 2004 (to 

2.6 credits). Because the total number of course credits earned by graduates increased over this 
period, average CTE credits between 1982 and 2004 did not decline as much as might be 
expected from the rise in new basics courses taken. Moreover, overall participation in CTE 
remained relatively stable, with over 95 percent of high school graduates earning some CTE 
credit between 1982 and 2004. 


Similarly, the proportion of high school graduates who are CTE investors (three or more CTE 
credits) declined by 11 points, from 46 percent to 35 percent between 1982 and 1992, but by 
2004 the proportion increased to 38 percent. 


CTE coursetaking became more diverse over these three decades. CTE explorers (three or more 
CTE credits in more than one occupational area) grew from 35 percent of investors in 1982 to 
55 percent of investors in 2004, while CTE concentrators (three or more credits on one 
occupational area) fell from 65 percent to 45 percent between 1982 and 2004. 


These trends coincided with CTE investors taking more advanced math and science courses and 
some improvement in their mathematics achievement. Among CTE investors, the proportion 
completing the new basics rose from 6 percent in 1982 to 58 percent in 2004. Further, the 
educational expectations and postsecondary education participation of CTE investors increased 
as well. Differences in initial postsecondary work experiences among CTE investors and non- 
investors did not substantially change. Those non-college-attending investors who had a first job 
were more likely to enter craft or skilled operative positions reflecting consistency with their 
CTE secondary school training. 


These changes are consistent with the direction of Perkins legislation for greater academic 
engagement by CTE students so that there are increasing similarities between CTE investors and 
non-investors in academic coursetaking. However, the nature of CTE investment is changing as a 
greater percentage of investors are taking their CTE courses in multiple occupational areas and 
thus exploring CTE rather than concentrating within a CTE occupational area. Explorers benefit 
from knowledge about several occupations, but they do not obtain the in-depth knowledge and 
skills of single-occupation concentrators. Secondary CTE performance reporting is also limited 
to reporting on concentrators who now constitute less than half the students who take three or 
more CTE courses. 
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3.10 Annex: Statistical Tables 


Exhibit 23. Percentage of public high school graduates, by CTE occupational investment: 1982, 


1992, and 2004 
CTE occupational investment 1982 1992 2004 
CTE investors 46.0 35.3 38.4 
CTE concentrators 29.7 19.8 17.3 
CTE explorers 16.3 15.5 21.1 
Non-investors 54.0 64.7 61.6 


Exhibit reads: The percentage of public high school graduates who were CTE investors was 46.0 percent in 1982. 


NOTE: CTE = career and technical education. CTE investors are the sum of concentrators and explorers. CTE concentrators 
earned three or more credits in at least one occupational area. CTE explorers earned three or more total occupational credits but did 
not concentrate in an occupational area. Non-investors did not earn three total occupational credits. A credit (Carnegie unit) is 
equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 24. Percentage of public high school graduates, by student characteristics and CTE 
investment: 1982, 1992, and 2004 


Student characteristic and CTE investment 1982 1992 2004 
Gender 
Male 

CTE investors 49.5 40.4 44.9 

CTE concentrators 31.0 22.6 21.0 

CTE explorers 18.5 17.8 23.9 

Non-investors 50.5 59.6 55.1 

Female 

CTE investors 42.9 30.3 32.4 

CTE concentrators 28.6 17.1 13.8 

CTE explorers 14.2 13.2 18.5 

Non-investors 57.2 69.7 67.6 


Race/ethnicity* 
Asian/Pacific Islander 


CTE investors 27.8 32.7 26.7 
CTE concentrators 13.0 16.8 9.6 
CTE explorers 14.7 15.8 17.2 

Non-investors 72.2 67.4 73.3 

Black 

CTE investors 46.9 32.8 38.4 
CTE concentrators 31.4 16.6 17.9 
CTE explorers 15.5 16.2 20.5 

Non-investors 53.1 67.2 61.6 

Hispanic 

CTE investors 53.6 32.8 33.5 
CTE concentrators 33.9 16.0 12.8 
CTE explorers 19.7 16.8 20.7 

Non-investors 46.4 67.2 66.5 

White 

CTE investors 45.0 36.1 40.5 
CTE concentrators 29.2 21.1 18.8 
CTE explorers 15.9 15.0 21.7 

Non-investors 55.0 63.9 59.5 


See notes at end of table. 


Continues next page 
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Exhibit 24. Percentage of public high school graduates, by student characteristics and CTE 
investment: 1982, 1992, and 2004—continued 


Student characteristic and CTE investment 1982 1992 2004 
Other 

CTE investors 52.4 41.7 36.7 

CTE concentrators 34.7 18.6 17.2 

CTE explorers 17.7 23.1 19.6 

Non-investors 47.6 58.3 63.3 


Socioeconomic status 
Lowest quartile 


CTE investors 57.7 52.1 44.2 
CTE concentrators 40.2 30.8 21.1 
CTE explorers 17.5 21.3 23.1 

Non-investors 42.3 47.9 55.8 


Middle two quartiles 


CTE investors 48.1 37.4 40.9 
CTE concentrators 31.6 21.8 18.4 
CTE explorers 16.5 15.6 22.5 

Non-investors 51.9 62.6 59.1 


Highest quartile 


CTE investors 29.0 21.5 28.6 
CTE concentrators 14.6 9.8 11.9 
CTE explorers 14.4 11.7 16.7 

Non-investors 71.0 78.5 71.4 


Exhibit reads: The percentage of male public high school graduates who were CTE investors was 49.5 percent in 
1982. 


* The category Asian or Pacific Islander includes Native Hawaiians. Hispanic may be of any race. Other category refers to those 
answering “other” in 1982 and 1992 and those answering “more than one race” in 2004. 

NOTE: CTE = career and technical education. CTE investors are the sum of concentrators and explorers. CTE concentrators 
earned three or more credits (Carnegie units) in at least one occupational area. CTE explorers earned three or more total 
occupational credits but did not concentrate in an occupational area. Non-investors earned less than three total occupational credits. 
A Carnegie unit is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 25. Percentage of public high school graduates completing given highest level of 
mathematics, by curricular foundation and CTE occupational investment: 1982, 1992, 


and 2004 
CTE investment and highest level of math completed 1982 1992 2004 
CTE investors 
No math or low academic math 34.9 19.4 7.5 
Algebra I/plane geometry 37.2 29.8 23.5 
Algebra II 15.6 26.8 30.0 
Algebra IIl/trigonometry/analytic geometry 9.1 13.8 15.9 
Precalculus/calculus 3.3 10.4 23.2 


CTE concentrators 


No math or low academic math 38.5 24.1 9.4 
Algebra I/plane geometry 38.2 31.6 24.7 
Algebra II 14.3 24.7 30.7 
Algebra Ill/trigonometry/analytic geometry 7.0 12.0 14.5 
Precalculus/calculus 2.0 7.6 20.6 


CTE explorers 


No math or low academic math 28.2 12.7 5.8 
Algebra I/plane geometry 35.3 27.5 22.4 
Algebra II 17.9 29.6 29.4 
Algebra Ill/trigonometry/analytic geometry 12.9 16.2 17.0 
Precalculus/calculus 5.6 14.0 25.3 


Non-investors 


No math or low academic math 17.7 8.0 4.3 
Algebra I/plane geometry 26.0 19.2 17.0 
Algebra Il 20.2 28.8 24.5 
Algebra IlIl/trigonometry/analytic geometry 20.6 17.6 18.3 
Precalculus/calculus 15.4 26.4 35.9 


Exhibit reads: The percentage of CTE investors who completed only “no math or low academic math” was 
34.9 percent in 1982. 


NOTE: CTE = career and technical education. CTE investors are the sum of concentrators and explorers. CTE concentrators 
earned three or more credits (Carnegie units) in at least one occupational area. CTE explorers earned three or more total 
occupational credits but did not concentrate in an occupational area. Non-investors earned less than three total occupational credits. 
A Carnegie unit is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 26. Percentage of public high school graduates completing given highest level of 
science, by curricular foundation and CTE occupational investment: 1982, 1992, and 


2004 
CTE investment and highest level of science completed 1982 1992 2004 
CTE investors 
No science or low-level science 22.8 4.1 3.4 
Secondary physical science and basic biology 17.2 10.2 4.6 
General biology 40.2 47.1 32.3 
Advanced biology, chemistry, or physics 19.7 38.6 59.7 


CTE concentrators 


No science or low-level science 25.7 4.9 5.0 

Secondary physical science and basic biology 18.6 10.6 5.8 
General biology 39.9 53.1 33.6 
Advanced biology, chemistry, or physics 15.9 31.5 55.6 


CTE explorers 


No science or low-level science 17.7 3.1 2.2 
Secondary physical science and basic biology 14.7 9.8 3.6 
General biology 40.8 39.4 31.2 
Advanced biology, chemistry, or physics 26.8 47.7 63.0 


Non-investors 


No science or low-level science 8.2 1.7 2.3 
Secondary physical science and basic biology 13.3 5.4 3.0 
General biology 30.4 29.9 22.3 
Advanced biology, chemistry, or physics 48.2 63.0 72.5 


Exhibit reads: The percentage of CTE investors who completed only “no science or low-level science” was 
22.8 percent in 1982. 


NOTE: CTE = career and technical education. CTE investors are the sum of concentrators and explorers. CTE concentrators 
earned three or more credits (Carnegie units) in at least one occupational area. CTE explorers earned three or more total 
occupational credits but did not concentrate in an occupational area. Non-investors earned less than three total occupational credits. 
A Carnegie unit is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Exhibit 27. Percentage of public high school graduates with given educational expectations in 
12th grade, by curricular foundation and CTE occupational investment: 1982, 1992, 


and 2004 
CTE investment and educational expectations 1982 1992 2004 
CTE investors 
High school or less 25.1 7.1 6.5 
Some college 48.5 37.8 25.0 
Bachelor’s degree 17.2 30.0 34.3 
Graduate or professional degree 9.3 19.2 25.7 
Don’t know — 6.0 8.5 


CTE concentrators 


High school or less 28.9 9.5 8.3 
Some college 49.6 40.5 27.9 
Bachelor’s degree 13.6 28.2 30.4 
Graduate or professional degree 7.8 14.9 24.0 
Don’t know _— 7.0 9.4 


CTE explorers 


High school or less 18.0 4.0 5.1 
Some college 46.4 34.3 22.7 
Bachelor’s degree 23.5 32.3 37.4 
Graduate or professional degree 12.0 24.6 27.1 
Don’t know — 4.8 7.8 


Non-investors 


High school or less 11.0 2.8 3.3 
Some college 31.3 20.6 14.6 
Bachelor’s degree 30.5 36.6 35.5 
Graduate or professional degree 27.2 35.7 40.4 
Don’t know — 4.3 6.2 


Exhibit reads: The percentage of CTE investors who expected to complete only high school or less was 25.1 percent 
in 1982. 


— Not available. 

NOTE: CTE = career and technical education. CTE investors are the sum of concentrators and explorers. CTE concentrators 
earned three or more credits (Carnegie units) in at least one occupational area. CTE explorers earned three or more total 
occupational credits but did not concentrate in an occupational area. Non-investors earned less than three total occupational credits. 
A Carnegie unit is equivalent to a course taken for one period each day for one full school year. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School and Beyond Longitudinal Study of 
1980 Sophomores (HS&B-S0:80/82), “High School Transcript Study”; National Education Longitudinal Study of 1988 (NELS:88), 
“Second Follow-up, Transcript Survey, 1992”; and Education Longitudinal Study of 2002 (ELS:2002), “First Follow-up, High School 
Transcript Study, 2004.” 
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Chapter 4. 


Outcomes of Career and Technical Education: 
Some New Evidence 


The mandate for the National Assessment of Career and Technical Education (NACTE) in the 
Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) includes an analysis 
of the impact that career and technical education (CTE) has on academic achievement and 
employment outcomes. Because of the time it takes for a newly authorized or reauthorized 
program to have an impact on students, NACTE will only be able to examine early outcome 
evidence associated with Perkins IV in its final report. Most of the studies and databases 
available for examining CTE impacts only include outcomes associated with the Carl D. Perkins 
Vocational and Applied Technology Education Act of 1990 (Perkins II) and the Carl D. Perkins 
Vocational and Technical Education Act of 1998 (Perkins IIT), and such is the case for the 
evidence presented in this chapter. The evaluators of the NACTE looked for additional ways to 
examine CTE outcomes and found two opportunities to utilize existing data that were available 
for this interim report. Our new evidence is based on a recent national longitudinal study of 
secondary students and a natural experiment involving choice of a CTE high school in 
Philadelphia.”® The final report will contain additional outcome studies. 


4.1 Context for NACTE Outcome Studies 


The context for NACTE’s investigations into the impact of enrollment in CTE on academic and 
employment outcomes was provided by the earlier mandated study of Perkins III, namely, the 
National Assessment of Vocational Education (NAVE; Silverberg et al., 2004; for a 1994 NAVE 
review, see Gamoran, 1998),7’ NAVE sponsored its own research into the value added by CTE 
under the Carl D. Perkins Vocational Education Act (Perkins I) and Perkins IT on a range of 
student outcomes and also conducted a review of the literature on CTE outcomes available at the 
time. 


At the secondary level, NAVE found that during the 1990s, students in CTE programs 
significantly increased their academic coursetaking and achievement. This finding is consistent 
with the evidence presented in Chapter 3 of this report. At the same time, NAVE also found that 
the research evidence consistently showed that enrollment in CTE programs did not bring about 
or contribute to those academic achievement gains. The larger context of those gains (also noted 
in Chapter 3) was the increase in high school graduation requirements among the states and the 
more intense focus on academic knowledge fostered by the ongoing education reform 
movement. 


°° This chapter was prepared by ED staffers Jay Noell (retired) and Michael Fong based on NACTE studies by 
Bozick and Dalton (2013) and Neild, Boccanfuso, and Byrnes (2013). 


*7 Rarlier forms of CTE were known as vocational education; the name was changed because CTE encompasses a 
broader approach to preparing students for further education, training, and careers rather than vocational education, 
which was more narrowly focused on providing job-specific skills. 
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NAVE also reported that only mixed evidence exists that enrollment in CTE reduces the high 
school dropout rate, with the most rigorous evidence suggesting that it did not. NAVE cites 
analyses of data from the National Education Longitudinal Study of 1988 as well as a natural 
experiment involving CTE programs in New York City (Crain et al., 1999) as having no impact 
or even a negative impact, which was the case in the latter study. 


Impact studies of career-related education and training programs found potentially positive 
results on some student outcomes. The evaluation of Career Academies looked at a CTE 
program model, a school-within-a-school, that had students take career-related curricula and 
related work experience through local employers. This study found positive effects on staying in 
and progressing through high school as well as earnings gains at the eight-year follow up for 
young men. The evaluation of Talent Development High Schools, a whole school reform with a 
“career academies” component in the upper grades, found positive effects on progressing 
through school. 


In moving to postsecondary attendance, NAVE found that the value added by CTE was 
negligible: CTE neither helped nor hurt the chances of students going on to postsecondary 
schooling. In the past, vocational education was criticized as handicapping students’ prospects of 
going on to postsecondary education, but current programs did not appear to have those negative 
consequences. 


Postsecondary vocational education did, however, provide significant economic returns. NAVE 
reported those earning a credential (certificate or degree) experienced the largest benefits. NAVE 
found in particular that CTE provided by community colleges appeared to produce substantial 
gains for the vast majority of participants. Those completing even as little as a year of 
postsecondary CTE courses, especially males, realized economic gains. 


4.2 NACTE Outcome Studies 


In developing its research agenda for examining the impact of CTE on student outcomes, 
NACTE looked for sources of existing data that could be used for analyzing CTE impacts. Four 
sources were identified. 


The first source was the Education Longitudinal Study of 2002 (ELS:2002), sponsored by the 
National Center for Education Statistics (NCES) in the U.S. Department of Education (this study 
was also discussed in Chapter 2). This longitudinal study continues the pattern NCES started of 
studying a cohort of secondary students every decade or so. This study began with high school 
sophomores in 2002, then followed up in 2004 with seniors, and then again in 2006, two years 
after the expected graduation date. The ELS:2002 study has transcript data so that the degree of 
participation in CTE can be measured along with secondary outcomes, postsecondary education 
and training, and employment history. 


The second source was administrative data in Philadelphia associated with the local high school 
choice program. Philadelphia has a fairly extensive high school choice program involving CTE 
as well as other (academic) high schools. Admission to CTE schools was basically made 
randomly, forming a natural experiment. That process allows studying how students randomly 
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selected into CTE high schools perform, compared with those not selected. The data available 
allow looking at the high school classes of 2003, 2004, and 2005. 


The third source was also city-level administrative data, but from San Diego, California. 
Although San Diego does have a choice program, the data for it have not been as readily 
accessible as in Philadelphia. But longitudinal data on students exist going back into elementary 
school. These data are being used to look at a variety of secondary and postsecondary outcomes 
for several cohorts of high school graduates. Results are not available for the interim report but 
will be available in time for the final report. 


The last source of data was Florida administrative data on the ninth-grade cohort of 1995—96, 
which was followed over 11 years. The entire cohort was followed through high school and then 
into postsecondary education and employment. The focus of this study is on postsecondary 
outcomes, in terms of education and employment. Results are not available for the interim report 
but will be available in time for the final report. 


4.3. Education Longitudinal Study of 2002 


ELS:2002 tracks students who were high school sophomores in 2002 and follows them into their 
postsecondary years. A nationally representative sample was drawn of 10th-graders in public and 
private schools in the United States in 2002 when Perkins III was in effect. Sample members 
were surveyed again in spring 2004, when most were high school seniors. In pring 2005, 
transcripts were collected from these students’ high schools. The data allow key student 
subgroups defined by the Elementary and Secondary Education Act of 1965 as amended (ESEA) 
to be examined as well as alternative ways of defining and analyzing CTE as occupational 
coursetaking. 


The ELS:2002 study measures student achievement in mathematics when they are sophomores 
and seniors and thus lends itself to evaluating how participation in CTE affects gains in math 
achievement between the sophomore and senior years. The study also is appropriate for looking 
at how CTE affects dropping out and completion of high school between the sophomore and 
senior years. Although relatively more students drop out of high school in or after the ninth grade 
than in other grades, dropping out between sophomore and senior years remains an important 
problem. Because ELS:2002 also gathers information on the type of high school students who 
attend, it can also be used to examine whether high school types—for example, an area CTE high 
school—helps or hinders secondary educational outcomes. 


Math Achievement 


NACTE’s study (Bozick and Dalton, 2013) analyzes gains in achievement in mathematics 
between the sophomore and senior years through the use of fixed effects regression analysis. This 
is a More rigorous approach than standard cross-sectional regression analysis that looks at 
student achievement at one point in time. Standard regression analysis can only suggest factors 
or variables that are associated with achievement but cannot identify causal effects because other 
unmeasured (sometime called unobserved) factors are not available for use in the analysis. Those 
factors may be associated with student self-selection into certain educational programs, making it 
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impossible to determine whether the program or some student characteristics leading to self- 
selection into the program were the causal factors affecting an outcome. 


Fixed effects regression analysis as used in this NACTE study (Bozick and Dalton, 2013) 
reduces unmeasured factors—such as attitudes, preferences, or innate ability—that are time 
invariant and may lead to self-selection into certain educational programs. Additional factors that 
vary over time also were added to the analysis. As a result, causal inferences about the efficacy 
of identified factors (such as CTE participation) can be made more confidently by using fixed 
effects regression, although estimates are not as unbiased as typically obtained from random 
experiments.”® 


In the fixed effects regression analysis, math achievement gains associated with CTE 
participation are measured in two ways (Bozick and Dalton, 2013). An overall test score is used 
to provide a general measure of math gains. In addition, five scores are calculated measuring the 
likelihood of mastery over specific and progressive benchmarks in mathematics ranging from 
basic to advanced mathematical proficiency. 


Participation in CTE was also measured two ways. First, overall CTE participation was 
measured in terms of the number of occupational courses taken. Family and consumer science 
courses and general labor force preparation courses were not included. Second, a more focused 
measure of occupational courses in science, technology, engineering, and mathematics (STEM) 
areas was used. 


The key findings from the study (Bozick and Dalton, 2013) follow. 


¢ The total number of occupational credits earned during the last two years of high 
school has no relationship to the increase in number of correct answers on the 
mathematics assessment. However, when occupational courses make up a larger 
percentage of the total number of courses taken, students answer fewer additional 
questions correctly on the mathematics assessment. 


Overall, there are no differences in mathematical proficiency gains between students who take a 
small number of occupational courses alongside their academic courses and students who take 
only academic courses. When students take relatively more occupational courses during the last 
two years of high school, however, they answer slightly fewer additional questions correctly on 
the mathematics assessment. This relationship remains statistically significant after controlling 
for a range of factors, but its magnitude is substantively small: about one-tenth fewer questions 
were answered correctly on the math assessment for each additional occupational course taken. 


* Gains in basic and intermediate mathematics skills and concepts are unrelated to 
occupational credits earned in the last two years of high school. However, taking 
relatively more occupational courses and fewer academic courses during the last two 


*8 Such causal inferences cannot be made with the same degree of confidence possible as when an experiment can be 
conducted in which students are randomly assigned to different programs, but that is not possible when examining 
many educational programs besides CTE. 
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years of high school limits the acquisition of advanced mathematical skills and 
concepts. 


The development of skills such as simple arithmetic operations, operations with decimals and 
fractions, and basic problem solving is not compromised by enrollment in occupational courses. 
The development of advanced mathematics skills such as solving multistep word problems is 
impeded when occupational courses make up a larger share of students’ course schedules. 
Though this relationship again remains statistically significant after controlling for a range of 
factors, its magnitude is substantively small. For example, those who earned at least eight and 
one-half credits in academic courses and two credits in occupational courses were less than a half 
percent less likely to be proficient at level 4, one of the most advanced skill levels, than those 
who earned 10.5 credits in academic courses and zero credits in occupational courses. 


* Occupational courses have similar effects on gains in mathematics achievement for 
both economically disadvantaged and affluent students as well as nonnative English 
speakers and native English speakers. Black and Asian students benefit more from 
occupational courses than do white students. 


Occupational courses improve the development of basic and intermediate math skills more for 
black students than for white students, while the development of intermediate and advanced math 
skills is fostered more for Asian students than for white students. 


¢ In general, STEM courses in the CTE curriculum neither enhance nor compromise 
overall math achievement. 


Improving learning in mathematics is largely a function of traditional academic mathematics 
courses. 


¢ Attendance at a full-time CTE school or a school located in a rural area is not related 
to mathematics gains. 


Effects of occupational coursetaking on math achievement gains do not vary by attendance at 
full-time CTE schools, but occupational coursetaking is less harmful to math achievement gains 
for students attending rural schools than for students attending suburban schools. 


The conclusion from the NACTE study of math gains (Bozick and Dalton, 2013) is that 
secondary CTE participation is largely unrelated to gains in mathematics proficiency between 
sophomore and senior years in high school. This new evidence from analyzing the ELS:2002 
data does not change the conclusion reached by the NAVE (Silverberg et al., 2004) report. 


Dropping Out of High School 


NACTE’s study of the relationship between CTE participation and dropping out of high school 
from the sophomore to senior years was also done with the ELS:2002 data (Bozick and Dalton, 
2013). Semester-by-semester dropout rates are generally low (2 percent or less). In the period 
analyzed over the last two years of high school, dropout rates increased each semester to the 
typical end of high school for most students (two years later). Then, in the fall semester 
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following the on-time graduation date for the cohort, over half of the remaining students dropped 
out. 


The analytic strategy used for examining dropping out was event history regression analysis. 
This approach predicts the odds of dropping out of high school, contingent on a set of 
independent factors or variables. A wide variety of student characteristics and some school-level 
characteristics were used in the analysis to isolate their effects. CTE and academic coursetaking 
patterns were calibrated to match the timing of possible dropping out. 


Key findings from the NACTE study of dropping out of high school (Bozick and Dalton, 2013) 
were the following: 


* Controlling for socioeconomic and academic differences among students as well as 
semester timing, the number of credits students earn in occupational courses is 
unrelated to their likelihood of dropping out. 


However, students who have accumulated relatively more credits in academic courses have a 
reduced likelihood of dropping out. On average, each additional academic credit earned lowers 
the odds of dropping out by 19 percent. 


¢ High school dropouts typically accumulate fewer academic credits than enrolled 
students over the same period of time; however, dropouts and enrolled students earn 
similar numbers of occupational credits. 


When comparing dropouts to enrolled students in any given semester, differences in the number 
of earned academic credits between students who drop out and students who remain enrolled are 
striking. However, no differences in occupational credits earned were observed. 


¢ The cumulative number of occupational credits relative to academic credits is positively 
associated with dropping out. However, this relationship is driven by low academic 
course completion among students enrolled in occupational courses. 


When occupational credits earned are considered relative to academic credits earned, a greater 
tilt toward occupational credits is related to a greater likelihood of dropping out. This 
relationship holds even when student socioeconomic and academic characteristics associated 
with dropping out are held constant. However, students with high numbers of academic credits 
and high occupational credits are no more or less likely to drop out than other students, 
indicating that accumulating a low number of academic credits drives the relationship between 
relative occupational coursetaking and dropping out. 


¢ Attendance ata full-time CTE school or a school located in an urban area is not 
related to dropping out. 


Effects of occupational coursetaking on dropping out do not consistently vary by full-time CTE 
school attendance or rural school attendance (where one might also expect a more positive 
influence of occupational courses on high school persistence). 


56 National Assessment of Career and Technical Education: Interim Report 


4.4 CTE Schools in Philadelphia: A Natural Experiment 


NACTE’s study of career and technical education high schools in Philadelphia used a natural 
experiment to examine the impact of CTE high schools on student outcomes, including 
achievement and completion (Neild, Boccanfuso, and Byrnes, 2013). These high schools have 
long been known as Area Vocational and Technical Schools. Three cohorts of their students 
were studied—the high school classes of 2003, 2004, and 2005, all of whom graduated when 
Perkins III was still in effect. The study involved four CTE schools whose primary focus was on 
providing career and technical training for students hoping either to pursue postsecondary 
education or enter the workforce immediately after high school. A fifth high school was given 
CTE status for the class of 2005. These high schools served students from across the district. 
Although the issue of attendance at CTE schools was also examined in the ELS:2002 study of 
achievement and dropping out discussed above (Bozick and Dalton, 2013), in which no positive 
effects were detected, most studies of CTE outcomes focus on coursetaking, not the effects of 
CTE schools. NACTE’s Philadelphia study examines the impact of CTE schools. 


The natural experiment in the impact of CTE schools occurred because many more students in 
each cohort applied to the CTE schools than could be accepted, and the CTE schools admitted 
students using a lottery that took into account only student race and ethnicity to achieve racial 
balance and how highly each student had prioritized the school on his or her application form.” 
For the class of 2003, CTE schools entered all applicants into their lotteries, regardless of their 
prior achievement, attendance, or behavior. For the classes of 2004 and 2005, each of the CTE 
schools prescreened applicants for school-related performance. After students with weaker 
records were removed from the applicant pool, the lottery was conducted. As a result of this 
prescreening, the CTE classes of 2004 and 2005 entered high school with stronger prior 
academic performance than the class of 2003. CTE coursetaking differed between those who 
attended CTE schools and those who applied but did not attend. Those attending CTE schools 
earned approximately twice as many CTE credits as those who applied but did not attend a CTE 
school. 


9 


Career and technical courses were offered at most of the approximately 40 public high schools in 
Philadelphia. (Other types of high schools besides CTE schools include neighborhood and 
special admission [magnet] schools, plus 17 charter schools.) Some of the larger neighborhood 
high schools had a variety of CTE courses of study that rivaled those offered in CTE schools. 
But, in general, the greatest variety of occupationally focused courses was found at CTE schools. 
CTE high schools operated within a district context that supported high school choice and that 
encouraged all high schools to offer college preparatory courses in mathematics, science, and 
foreign languages. 


Research Questions 


The Philadelphia study addressed the following three key questions: 


°° All analyses for results reported below controlled for student race and ethnicity because empirical analysis 
showed the probability of acceptance at particular CTE schools in particular cohorts did vary according to racial and 
ethnic background. 
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¢ What is the effect of CTE schools on educational attainment, specifically, credit 
accumulation, grade promotion, and graduation? 


¢ What is the effect of CTE schools on college preparatory coursetaking in mathematics, 
science, and foreign language? 


¢ What effect do CTE schools have on academic performance, specifically, academic 
growth in mathematics and reading comprehension? 


The natural experiment allowed two types of estimates of the impact of CTE schools to be 
calculated. The first is called an intent-to-treat (ITT) estimate, which is conventionally used in 
experiments. It addresses the question of how the outcomes of students who were accepted to 
CTE schools differed from the outcomes of students who were not accepted. Because some of 
the students accepted may not have attended, a second estimate is often calculated in 
experimental studies that takes participation or exposure into account. This is called a dosage 
estimate (DE), and it estimates the association of actual participation in CTE schools with 
outcomes. 


This study relies entirely on student record data kept by the School District of Philadelphia. 
These data include enrollment and graduation status, transcript information (including course 
grades), test scores, attendance, special education status, English-language-learner status, and 
school(s) attended, from eighth grade forward, in addition to applications to high schools and 
high school admissions decisions. 


Key Findings on the Impact of CTE Schools 


The findings from the study represent those from one large urban community with large minority 
concentrations. The study also involves students who were motivated to apply to CTE schools. 
Therefore, these findings may not generalize to other community types or to other students. In 
addition, the study examined selective CTE schools for the classes of 2004 and 2005, where 
students applied to a CTE school and passed some moderate screens for achievement, 
attendance, and behavior. Therefore, findings for the classes of 2004 and 2005 cannot be 
generalized to other CTE school models that are not selective. 


¢ Attending a CTE high school increased the probability of students successfully 
completing the college preparatory mathematics sequence of algebra I, algebra I, and 
geometry. 


The ITT estimates placed the odds of completing this course sequence as one-quarter to one-third 
greater for students in CTE schools (25 percent to 32 percent increase in the odds of completing 
the sequence), while the DEs placed the odds for those attending CTE schools between two and 
three times greater than those who attended other schools (232 percent to 255 percent increase in 
odds of completing mathematics course sequence). 


¢ Attending a CTE high school had no relationship to math achievement growth from 
eighth to 11th grades. 
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The CTE effect for learning growth in mathematics was generally statistically insignificant, and 
the effects were always small. 


¢ Attending a CTE high school had no relationship to reading achievement growth from 
eighth to 11th grades. 


The CTE effect for learning growth in reading comprehension was generally statistically 
insignificant, and the effects were always small. 


¢ Attending a CTE high school improved on-time graduation rates in each of the three 
cohorts. 


This CTE advantage in on-time completion continued to five-year graduation rates for the two 
cohorts for which data were available and to the six-year graduation rates for the one cohort for 
which data were available (13 percent to 27 percent increase in the odds of graduating using ITT 
estimate, and 111 percent to 183 percent increase in the DE). Likewise, there were CTE impacts 
on total credits earned (0.7 credits to 0.8 credits with ITT estimate, and 5.9 credits to 6.6 credits 
with DE) and total CTE courses taken (0.33 to 0.38 with ITT estimate, and 2.30 to 2.34 with 
DE). 


¢ Estimates of impact of attending CTE schools on completing both chemistry and 
physics credits were inconsistent across cohorts and often not statistically significant. 


¢ Estimates of impact of attending CTE schools on earning two course credits in foreign 
language were inconsistent. The DE but not the ITT indicated an impact on completing a 
foreign language course sequence (with those attending CTE schools having over twice 
the odds of successfully completing two years of a foreign language). 


4.5 Conclusion 


When the NACTE studies of CTE were first initiated, the context for examining the outcomes of 
CTE came largely from the conclusions of the NAVE final report and other recent studies of 
CTE-related programs. As such, the NACTE team looked for opportunities to clarify, extend, 
and deepen the research based on CTE outcomes. Studies of CTE can examine quite different 
treatments and can vary by research methodology and population. The NACTE commissioned 
studies that examined differences in student outcomes using two different program approaches. 
The ELS:2002 study looked at the effects of CTE coursetaking, in particular students who take 
more versus fewer CTE courses in traditional high schools, while the Philadelphia study 
examined the effects on students of winning admission to a selective CTE high school versus 
another type of high school where they might also take CTE courses. 


Two additional NACTE studies on the effects of CTE coursetaking are also being examined. 
One analyzes the administrative data available in San Diego for high schools and colleges; and 
the other capitalizes on the administrative database spanning secondary and postsecondary levels 
in Florida. This interim report has findings to date from the ELS:2002 and Philadelphia studies, 
with additional follow-up to be reported on every study in the final report. 
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NACTE’s findings from ELS:2002 study are consistent with the conclusions reached by NAVE. 
At the secondary level, evidence suggests that CTE coursetaking does not bring about or 
contribute to academic achievement gains nor does it reduce the high school dropout rate. 
Evidence from a natural experiment in Philadelphia found a positive relationship between CTE 
school attendance in oversubscribed schools and course and school completion but no 
relationship between CTE school attendance and achievement. 
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Chapter 5. 


Career and Technical Education 
in International Perspective 


In reauthorizing the Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV), 
Congress added to section 2 a new purpose for career and technical education (CTE)— 


(7) providing individuals with opportunities throughout their lifetimes to develop, in 
conjunction with other education and training programs, the knowledge and skills needed 
to keep the United States competitive [italics added]. 


To help maintain and improve the competitiveness of the United States, it is useful to compare 
and understand how the CTE provided by U.S. competitors differs from that in the United States. 
The importance of learning from the practices of other education systems to keep the United 
States competitive is increasingly being recognized. The Congress has also expressed interest in 
international comparisons under section 114(a)(3) of Perkins IV, which provides that National 
Center for Education Statistics (NCES) “may include international comparisons” in its 
assessments of CTE. In Benchmarking for Success: Ensuring U.S. Students Receive a World- 
Class Education, three leading educational policy organizations—the National Governors 
Association (NGA), Council of Chief State School Officers and Achieve, Inc.—call for 
benchmarking U.S. education systems against those of top-performing nations. 


This chapter? provides an introduction to CTE—more widely known internationally as technical 
and vocational education and training, or simply as vocational education—in nations that are 
economic competitors to the United States and are members of (or participants in data collections 
sponsored by) the Organization for Economic Cooperation and Development (OECD).”! 


The analysis of OECD data addresses two questions: 


* How does the mix of skilled and unskilled workers in the U.S. workforce compare with 
the mix in other OECD countries? 


* How does the preparation of secondary CTE students in vocational education compare 
with the preparation in other OECD countries? 


The OECD data used here and published in its report Education at a Glance 2008 provide a basis 
for initiating a discussion of what the United States needs to learn from its close competitors 
about providing CTE. Further research will be needed for achieving a full understanding of the 
different approaches to vocational education taken by other nations and of their applicability to 
the U.S. CTE system. 


* This chapter was prepared by ED staffers Jay Noell (retired) and Michael Fong. Sam Nayman, U.S. Department 
of Education intern, provided research assistance. 


*! Unfortunately, China and India are not members of OECD and have not been participating in OECD data 
collections for the Education at a Glance series from which this chapter draws. 
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The OECD data used in the chapter have strengths and weaknesses. A strength of the data is that 
they are collected from a range of developed nations using standard definitions and terminology, 
thus permitting comparative analyses. But capturing the unique features of a nation’s educational 
system using terms designed to be universally applicable often means missing important 
subtleties about how that system really works. Hence, international comparisons from large-scale 
studies are only suggestive of promising practices and require further in-depth analysis; 
ultimately, actual validation occurs only by evaluating promising practices within the U.S. 
context. In some instances, it also means that a nation (sometimes the United States) cannot 
provide the data requested by OECD because its data system uses very different definitions and 
produces data that cannot be easily mapped into OECD categories. 


5.1 OECD Nations 


Vocational education in 15 OECD nations and CTE in the United States are compared. The 
nations were identified as top competitors to the United States in The Global Competitiveness 
Report, 2009-2010, issued by the World Economic Forum (Schwab, 2009), and had data 
reported in OECD’s Education at a Glance 2008 report (see Section 5.7 for more detail on the 
included nations). 


According to The Global Competitiveness Report, 2009-2010 (Schwab, 2009), 13 of the nations 
compared with the United States in this chapter are among the top 20 nations most competitive 
with the United States, while the other two, Italy and the Russian Federation, are ranked 48 and 
63, respectively (see Exhibit 31). The nations identified by the World Economic Forum were 
linked with the nations included in the OECD report (2008) that have relatively complete data on 
their vocational education systems.” 


5.2 Labor Market Context 


Before examining the OECD education data to compare CTE in the United States with the 
vocational education provided in the 15 selected competitor countries, it is worth briefly 
examining the skill-level distributions of occupations in the economies of these countries. 
Occupational skill-level distributions are important because education systems need to provide 
new (and upgrade continuing) workers with the requisite knowledge, skills, and abilities needed 
to successfully perform jobs requiring different levels of skill. It is likely that some of the 
differences in the characteristics of the education systems among nations are related to 
differences in the occupational skill-level distributions in their economies. Some countries will 
have more—and others fewer—highly skilled jobs, for example, and this could affect the kinds 
of education offered in a country. 


*° Among other top 20 U.S. competitors, Singapore (3), Hong Kong (11), and Taiwan (12) are not further compared 
in this chapter because they do not participate in the OECD survey; Denmark (5), Norway (14), and New Zealand 
(20) are not included because key data on their vocational education systems are missing. 
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The occupational skill-level distributions in the United States and the 15 selected comparison 
nations are found in Exhibit 28. The OECD uses the International Standard Classification of 
Occupations (ISCO) in defining skilled jobs (those at what are defined in ISCO as skill levels 3 
and 4), semiskilled jobs (skill level 2), and unskilled jobs (level 1). Because of missing data (e.g., 
the United States did not provide separate data on semi-skilled and unskilled jobs), occupations 
in Exhibit 28 are divided into two categories: skilled jobs, and semiskilled and unskilled jobs. 


¢ Skilled jobs encompass professionals, managers, technicians, and associate professionals. 
They require at least some tertiary education (usually called postsecondary education in 
the United States). For example, jobs at skill level 4 require a degree, in the United 
States, at the bachelor’s level or above, while level 3 requires completion of at least a 
short or medium tertiary program. Sample skilled occupations include engineers, 
secondary school teachers, computer systems analysts, shop managers, medical 
laboratory technicians, and broadcasting and recording technicians. 


¢ Semiskilled jobs typically involve the performance of tasks such as operating machinery 
and electronic equipment, driving vehicles, maintaining and repairing equipment, and 
manipulating, ordering, and storing information. They require at least the completion of 
lower secondary education (as defined by the 1997 revision of the International Standard 
Classification of Education [ISCED97] level 2, or grades 7 through 9 in the United 
States), and many require completion of upper secondary schooling (ISCED97 level 3, or 
grades 10—12 in the United States), including a significant component of vocation- 
specific education undertaken after the completion of secondary education (ISCED97 
level 4). Sample occupations include account clerks, machinists, police officers, 
hairdressers, electricians, and mechanics. 


¢ Unskilled jobs, which typically constitute less than 10 percent of all occupations in 
OECD countries, require performance of simple and routine physical or manual tasks. 
Completion of primary education (ISCED level 1, or grades 1 through 6 in the United 
States) may be required, plus possibly some on-the-job training. Sample occupations 
include office cleaners, freight handlers, gardeners, and kitchen assistants. 


Between 1998 and 2006, the share of jobs in the skilled category increased in the United States 
and all competitor countries except the Netherlands. Among all OECD countries, almost 

4 percent more workers were in skilled occupations, and about 4 percent less were in semi- 
skilled jobs in 2006 than in 1998. OECD (2008) identifies technicians, in particular, as the 
largest occupational group in 2006. Technicians are in the skilled occupation category and now 
outnumber those in craft and related trades, which are in semiskilled occupations and were the 
largest occupational group in 1998. Note that the OECD advises caution in interpreting these 
figures since some countries have revised their ISCO classification in the past several years. 
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Exhibit 28. Proportion of 25- to 64-year-olds in the workforce, by skill level of occupation in 
selected countries: 2006 


Semi- and Change in percent 
Country/economy Total (%)* Skilled (%) unskilled (%) skilled, 1998 to 2006 
United States 100 39 61 2 
Australia 100 51 50 — 
Austria 100 40 60 7 
Belgium 100 46 54 4 
Canada 100 46 54 3 
Finland 100 48 52 4 
France 100 42 58 5 
Germany 100 44 56 4 
Italy 100 41 59 11 
Japan 100 — — — 
Korea 100 — — — 
Netherlands 100 53 47 -1 
Sweden 100 46 55 3 
Switzerland 100 49 52 4 
United Kingdom 100 44 56 1 
Russian Federation 100 — — — 


Exhibit reads: The percentage of U.S. 25- to 64-year-olds in the workforce who are in skilled occupations is 39 
percent. 


* Totals may not add to 100 because of rounding. 
— Not available or not applicable. 
SOURCE: Organization for Economic Co-operation and Development (OECD), Education at a Glance 2008 (2008, Table A1.6). 


Perhaps the most interesting data in Exhibit 28 concern the relative proportions of workers in 
skilled and semiskilled and unskilled occupations in the United States, compared with its 
competitor nations. Among the reporting countries, the United States has the smallest share of 
workers in skilled occupations, at 39 percent, and the largest share in semi-skilled and unskilled 
jobs, at 61 percent. The median share of skilled jobs among competitor countries is 46 percent, 
and at the upper end the Netherlands reports 53 percent and Switzerland 49 percent skilled 
occupations (with the converse being the share of workers in semiskilled and unskilled 
occupations). 


The relative proportions of workers in skilled versus semi-skilled or unskilled occupations have 
implications for the kinds of education needed for preparing students for the workforce. Since 
skilled occupations typically call for tertiary education, often at the level of the bachelor’s degree 
in the United States, while semiskilled occupations generally require completion of upper 
secondary schooling (a high school diploma) or some nontertiary postsecondary training, one 
might expect to see differences among countries in the degree to which they enroll students in 
tertiary education programs leading to skilled occupations. 


5.3. Upper Secondary Education 


OECD uses the classification system and definitions for education developed in ISCED97. The 
basic categories relevant for vocational and career and technical education are upper secondary 
and tertiary education levels. 
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Secondary education (which begins after six years of primary education) encompasses two 
levels: lower secondary education and upper secondary education. 


¢ Lower secondary education (which typically takes three years) essentially completes a 
basic education and coincides with the end of compulsory education in many economies. 
In the United States, lower secondary education corresponds to grades 7 through 9. 


¢ Upper secondary education (which typically also lasts three years) provides students with 
more in-depth knowledge to prepare them for tertiary education or entry into the labor 
market. In the United States, upper secondary corresponds to grades 10-12. 


Upper secondary education (level 3 in the ISCED97 categories) encompasses three types of 
program orientation— 


* General education (3A) provides students with a deeper understanding of a subject (or 
group of subjects), especially to prepare them for further education at a higher (or the 
same) level (e.g., tertiary education). Such programs are typically school-based and may 
or may not provide participants with a labor-market-relevant qualification. 


¢ Pre-vocational or pre-technical education (3B) is intended to introduce students to the 
world of work and prepare them for entry into vocational or technical education 
programs. Successful completion generally does not lead to a labor-market-relevant 
vocational or technical qualification, but such education must have at least 25 percent of 
its content be vocational or technical. 


¢ Vocational or technical education (3C) is designed to have students acquire practical 
skills, know-how, and understanding necessary for employment in a particular occupation 
or trade or class of occupations or trades. Successful completion leads to a labor-market- 
relevant vocational qualification recognized by competent authorities in the country in 
which it was obtained (e.g., ministry of education, employers’ associations). Some 
countries combine school- and work-based learning in their vocational programs, and 
OECD classifies a program as school-based if at least 75 percent of the curriculum is 
presented at school. 


Upper secondary school enrollment patterns of the United States and its competitor countries are 
shown in Exhibit 29. Because the United States does not have routine data collections including 
information on vocational programs at the upper secondary school level meeting the ISCED97 
definitions, the proportion of secondary students whose curriculum is made up of at least 

25 percent vocational or technical (occupational) courses must be estimated using special 
periodic data collections. In this chapter we use the Education Longitudinal Study of 2002, 
sponsored by the National Center for Education Statistics in the U.S. Department of Education, 
which collected transcript data for high school graduates in 2004. Among those 2004 high school 
graduates, about 6 percent earned 25 percent or more of their credits in CTE occupational 
courses (for more detail, see Chapter 3 above), the share of the curriculum specified in ISCED97. 
This figure is used in Exhibit 29 instead of the number reported in Education at a Glance 2008 
(OECD, 2008). 
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Exhibit 29. Upper secondary school enrollment distribution in the United States and selected 
other countries, by program orientation percentage: 2006 


Country/economy 
United States 
Australia 

Austria 

Belgium 

Canada 

Finland 

France 

Germany 

Italy 

Japan 

Korea 

Netherlands 
Sweden 
Switzerland 

United Kingdom 
Russian Federation 


Total 


100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 
100 


General 


94 
38 
22 
31 
95 
35 
57 
A 
40 
75 
72 
33 
45 
36 
58 
56 


Vocational, 
total 


44 


Vocational 
type: Pre- 
vocational 


6 


14 


Vocational 


type: 
Vocational 


62 


30 


Combined 
school- 
and work- 
based 


Includes 
enrollment in 
formal 
apprenticeships 
No 

Yes 

Yes 

Yes 

Yes 


Exhibit reads: The percentage of U.S. upper secondary school students in a general education program is 


94 percent. 


— Not available or not applicable. 


SOURCES: Organization for Economic Co-operation and Development (OECD), Education at a Glance 2008 (2008, Table C1.1, 
p. 326); U.S. estimate based on National Center for Education Statistics’ Education Longitudinal Study of 2002 for 2004 high school 


graduates. 


OECD data show that secondary students in competitor nations devote a higher share of their 
course work to secondary vocational education programs than the United States, with the 
exception of Canada (see Exhibit 27). Using OECD standards of vocational education 
enrollment, in most European countries examined, almost half or more of secondary school 
graduates enroll in a vocational-oriented program (in which they earn 25 percent or more of their 
total credits). Even in Korea and Japan, with their strong academic traditions, a quarter of 
secondary graduates concentrate in vocational education by OECD standards. By contrast, since 
the early 1980s the percentage of U.S. secondary students meeting OECD standards declined 


from about 18 percent to 6 percent. 


It is worth noting that 25 percent of total credits earned in CTE in the United States is well above 
the definition of three CTE credits in one occupational area used to define a CTE concentrator 
for Perkins IV accountability reporting. The OECD definition of a vocational education 
concentration would be equivalent to five or six CTE credits in the United States, which is more 
typical of U.S. career academy models or programs with an apprenticeship or internship 
component. The OECD data raise but cannot answer the question of the likely effectiveness for 
U.S. secondary students taking these types of CTE programs. 
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Among U.S. competitors, 9 of 15 have a majority of their students enrolled in vocational 
(including prevocational) programs instead of general education programs. Besides the United 
States, exceptions include Canada, France, Japan, Korea, the United Kingdom, and the Russian 
Federation. However, only Canada is similar to the U.S. in that it has a percentage of students 
enrolled only in vocational education programs around the 5 percent level. 


With the exception of Italy and the Russian Federation, most of the students in vocational 
programs are in programs leading to a labor-market-relevant credential (or an ISCED97 3C 
vocational program; enrollment in 3B pre-vocational programs is relatively low). 


Among countries with a majority of students in upper secondary vocational programs, six have at 
least 10 percent of enrollment in programs jointly school- and work-based. In Germany and 
Switzerland, for example, most vocational students are in such programs. 


All the countries in Exhibit 29, with the exception of the United States, Japan, Korea, and 
Sweden (and possibly Italy), have apprenticeship programs that are a recognized part of their 
educational system. The United States does have apprenticeship programs, but they are not part 
of the formal education system (and, at the federal government level, are coordinated through the 
U.S. Department of Labor).* In some countries (e.g., Austria and Germany), students are 
responsible for securing an apprenticeship. Across the countries with apprenticeship programs 
(which start after completion of lower secondary schooling), the duration ranges from one to four 
years. Successful completion of the apprenticeship usually results in the awarding of an upper 
secondary qualification (or, in some cases, a postsecondary, nontertiary one”). 


5.4 Upper Secondary School Completion Requirements, by Program 
Orientation 


Student performance is sometimes measured by the number of required courses or hours of 
instruction in a program that a student must take to complete a program and earn a credential, 
diploma, or degree. And, sometimes, it is reflected in the use of assessments or tests at various 
points in a program, including at the end, where a test may be used to ensure that a program 
completer has met at least minimum standards of competence or proficiency, as measured by a 
test (an exit test) before a credential, diploma, or degree is awarded. 


There is particular concern in many countries about the preparation of general versus vocational 
programs at the upper secondary school level. In some countries, vocational programs are seen as 
less rigorous than general programs, and there is a concern that this shortchanges students by not 
adequately preparing them for career and life success. Students need to be equipped with the 
skills that will allow them to continue learning throughout their lifetimes so that they may adapt 


*° The School-to-Work Opportunities Act of 1994 supported developing apprenticeships as part of local education 
systems in the United States but was not reauthorized. 


* The ISCED97 categories also include level 4, postsecondary nontertiary education programs that straddle 
secondary and tertiary levels. In some countries, such programs are considered secondary; in others, they are 
considered postsecondary. These programs are primarily vocational or technical in nature. In the United States, such 
programs often take the form of vocational certificate programs (often of a year or less duration) offered by 
community colleges or proprietary educational institutions. Because OECD data are sparse for this level, they are 
not considered here. 
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to changing economic and occupational demands. Such learning might occur immediately after 
secondary schooling, through tertiary education or training, or it might occur later, after some 
experience is acquired in the labor market. But students need to have mastered the foundational 
skills and knowledge necessary not only for immediate entry into the labor market but also for 
being able to successfully continue into tertiary schooling. 


Among competitor countries to the United States, the variety of approaches taken to ensure 
appropriate rigor in terms of upper secondary education program completion requirements is 
shown in Exhibit 30 by type of program orientation (general vs. vocational). 


The options include the use of a final exam, a series of exams during the program, or a specified 
number of required course hours.” It can also be noted that only two of these countries (France 
and Italy) have national examinations in lower secondary general education programs. 


Regarding upper secondary program completion requirements in countries having both general 
and vocational (which includes prevocational) programs, the strategy to ensure program rigor 
through program completion requirements is typically identical. If a general program has a final 
exam, the vocational program has a final exam. Similarly, if the completion requirement for a 
general program is in terms of number of courses, a similar requirement will hold for vocational 
programs. 


In terms of the specific strategies used, only the United States and Korea rely on a specified 
number of course hours for students to complete their programs and earn an upper secondary 
credential. In the United States, those requirements are generally specified in terms of Carnegie 
units (i.e., completion of one hour per day for two semesters to earn one credit). 


Competitor countries to the United States, except for Japan and Korea, widely rely on exams to 
ensure rigor: either final exams or a series of exams during the program. Some of the states in the 
United States do require exams—either final (school exit) exams or during the program (end-of- 
program exams)—and some do not. A similar situation appears in Australia, which also has a 
federal structure, with territories as well as states. 


Most competitor countries examined in Exhibit 30 require a final exam. However, in some 
countries—such as Finland and Sweden—not all programs do. But a number of countries rely on 
both a series of exams during the program as well as a final exam. These include Belgium, 
Germany, the Netherlands, and Switzerland. 


35 ‘ ‘ : : Z : . 
An option pursued by some countries—including Japan—is to combine course hour requirements and exams. 
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Exhibit 30. Completion requirements for upper secondary school programs in the United States 
and selected other countries, by program orientation: 2006 


Vocational 
General General Vocational programs: 
programs: programs: programs: Only 
Series of Only specified Vocational Series of specified 
General exams number of programs: exams number of 
Country/ programs: Final during course Final during course 
economy exam? program? hours? exam? program?* hours?* 
United States 20 states Y/30N Somestates Y — — — 
Australia NY? Y N N Y N 
Austria Y Y N Y/N® Y N 
Belgium Y Y N Y Y N 
Canada-Quebec® N Y N N Y N 
Finland Y/N Y N — — — 
France Y N N Y/N Y — 
Germany Y Y N Y Y N 
Italy Y N N Y Y/N° N 
Japan N N N N N N 
Korea N N Y N N Y 
Netherlands Y Y N Y Y N 
Sweden Y/N Y/N Y/N — — — 
Switzerland Y Y — Y Y — 
United Kingdom N° Y N — Y N 


Russian Federation — = = _ = = 


Exhibit reads: Twenty of the States in the United States require a final exam in order to complete a general upper 
secondary school program. 

Y =yes; N =no. 

— Not available or not applicable. 

* Includes prevocational programs. 

: Completion requirements vary by state or territory. 

° Some vocational and prevocational programs have different requirements. 

* This information about Canada only applies to Quebec. 

° There is usually no final examination, but some programs can be completed that way. 

SOURCE: Organization for Economic Co-operation and Development (OECD), Education at a Glance 2008 (2008, Table X1.3). 


5.5 Career and Technical Education in International Perspective 


In assembling data published by OECD in Education at a Glance 2008 on career and technical 
education in the United States and 15 of its competitor countries, this chapter has identified a 
number of contextual findings, including— 


¢ The share of skilled occupations increased in all but one competitor country between 
2000 and 2006, but the United States has relatively fewer workers in skilled occupations 
and more semi-skilled and unskilled occupations than the 15 competitor nations 
examined. Further analyses are needed to assess whether this reflects a lower supply of 
skilled workers or a different occupational structure. 


* Based on OECD’s definition of a secondary vocational education program, which has at 
least 25 percent of total course work as being vocational or technical in content, the 
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United States at 6 percent is near the international bottom in its small share of secondary 
students enrolling in a greater proportion of vocational education course work compared 
with OECD countries. Moreover, nine of 15 competitor nations have more students 
enrolled in secondary vocational programs than in secondary general programs. 


¢ Among countries with expenditure data at the secondary level, vocational education is 
typically more expensive to provide than general education. Seven of the eight 
competitor nations reporting data spend more on vocational education than on general 
education. 


* OECD countries typically use the same completion requirements (e.g., exams or course 
hours) to ensure program rigor in secondary general and vocational programs (although, 
e.g., the exams or courses would not necessarily be identical). 


— Most competitor nations use final exams or a series of exams during secondary 
educational programs to ensure program rigor and the competency level of 
completers, but such practices are found in only about half of the states in the United 
States. 


— Only the United States and Korea address upper secondary program completion 
requirements primarily through a specified number of course hours (usually in 
Carnegie units in the United States). 


5.6 Annex: America’s Global Competitors 


The World Economic Forum approach to calculating economic competitiveness examines the 
institutions, policies, and factors that determine the level of productivity of a country or economy 
because productivity sets the sustainable level of prosperity that can be earned by an economy 
(Schwab, 2009). In this perspective, productivity not only determines a country’s level of 
income, it is also central to determining an economy’s growth potential. 


In examining a nation’s competitiveness, education is a central factor, or as the World Economic 
Forum terms it, “pillar of competitiveness” (Schwab, 2009). Today’s globalizing economies need 
to nurture pools of well-educated workers who are able to adapt rapidly to their changing 
technical and economic environments. More specifically, education and training are seen as 
efficiency-enhancing factors interacting with and supporting “innovation and [business] 
sophistication” factors that are critical if a nation is to remain among the very most competitive 
economies. 


°° The World Economic Forum classifies “higher education and training” as one pillar of competitiveness and 
“health and primary education” as another pillar. In its terms, higher education and training covers secondary and 
tertiary (the latter, in U.S. terms, postsecondary) enrollment as well as the quality of education as assessed by the 
business community. The forum also notes the importance of vocational and continuous on-the-job training—which 
is neglected in many economies—for ensuring a constant upgrading of workers’ skills to the changing needs of the 
evolving economy (Schwab, 2009). 
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As Exhibit 31 below indicates, the United States is identified as the world’s second most 
competitive economy in 2009-10, behind Switzerland and down from being the most 
competitive nation in 2008-09. According to Schwab (2009), the United States also does not 
have the top-rated pillar for higher education and training (which covers secondary and 
tertiary—in the United States, postsecondary—levels of education). That distinction is held by 
Finland, while the United States ranks 7. Countries ranking higher besides Finland are Denmark 
(2), Sweden (3), Iceland (4), Singapore (5), and Switzerland (6). 


A strong association exists between overall global economic competitiveness and the strength of 
the higher education and training pillar. The rank correlation between the overall Global 
Competitiveness Index score and the higher education and training pillar score among the 

22 countries in Exhibit 32 is over 0.8 (on a scale of zero to one), meaning an overlap in variation 
of over 65 percent between overall competitiveness and strength of higher education and 
training. This suggests that while overall competitiveness depends on other factors, the higher 
education and training pillar (covering secondary and tertiary education levels) is clearly 
associated with maintaining and improving a nation’s overall competitiveness. 
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Exhibit 31. Global Competitiveness Index (GCI) rankings of selected nations: 2009-10 and 


2008-09 
Higher Higher 
education and education and 

Countries/economies Overall Overall training pillar —_ training pillar Overall 
examined using rank, score, rank, score, ranking, 
OECD education data 2009-10 2009-10 2009-10° 2009-10° 2008-09 
Switzerland 1 5.60 6 5.60 2 
United States 2 5.59 7 5.57 1 
Sweden 4 5.51 3 5.76 4 
Finland 6 5.43 1 5.97 6 
Germany 7 5.37 21 5.15 7 
Japan 8 5.37 23 5.06 9 
Canada 9 5.33 9 5.50 10 
Netherlands 10 5.32 10 5.49 8 
United Kingdom 13 5.19 18 5.27 12 
Australia 15 5.15 14 5.33 18 
France 16 5.13 15 5.30 16 
Austria 17 5.13 17 5.19 14 
Belgium 18 5.09 8 5.52 19 
Korea (Rep.) 19 5.00 16 5.24 13 
Italy 48 4.31 49 4.35 49 
Russian Federation 63 4.15 51 4.30 51 
Others? 

Singapore 3 5.55 5 5.62 5 

Denmark 5 5.46 2 5.90 3 

Hong Kong, SAR 11 5.22 31 4.74 11 

Taiwan, China 12 5.22 10 5.52 17 

Norway 14 5.17 12 5.48 15 

New Zealand 20 4.98 4 5.69 23 


Exhibit reads: Switzerland was ranked first overall on the GCI for 2009-10. 


* Schwab (2009) defines this higher education pillar or component index as spanning secondary and tertiary (in the U.S., 
postsecondary) education (primary—or, in the U.S., elementary—education is included with health in a separate pillar). The higher 
education pillar measures secondary and tertiary enrollment rates as well as the quality of education as assessed by the business 
community; the extent of training is also factored in because of the importance of vocational and continuous on-the-job training for 
ensuring workers’ skills are upgraded to the changing needs of the evolving economy. 

» Singapore, Hong Kong, and Taiwan are not further compared in this chapter because they do not participate in the Organization 
for Economic Co-operation and Development (OECD) survey; neither do China (ranked 29) and India (49). Denmark, Norway, and 
New Zealand are not used because key data on vocational education are missing. 

SOURCE: Schwab, K. 2009. The Global Competitiveness Report 2009-2010. Geneva, Switzerland: World Economic Forum. 
Downloaded October 30, 2009, from http://www.weforum.org/pdf/GCR09/GCR2009201 Ofullreport.pdf. 
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Chapter 6. 


Implications and Final Report 


This interim report is the first of two reports to Congress that make up the National Assessment 
of Career and Technical Education (NACTE). The report lays out a framework for evaluating 
state and local implementation efforts under the Carl D. Perkins Career and Technical 
Education Act of 2006 (Perkins IV) and in measuring student successes in career and technical 
education (CTE). The purpose of the interim report is to describe the research approach and 
present findings to date from the national assessment. The interim report provides baseline 
information and builds a foundation for the more comprehensive final report. 


6.1. Implications 


These interim report findings about CTE participation, effectiveness, and international 
benchmarking with the Organization for Economic Cooperation and Development (OECD) 
countries raise questions that warrant further exploration in the final NACTE report and beyond. 
These issues include: 


¢ How to assess the outcomes of the increasing proportion of students who invest in CTE 
courses (take three or more credits) but take fewer than three credits in a single 
occupational area (explorers). CTE explorers are not part of the current Perkins 
performance accountability system yet they are a growing majority of CTE investors. 
Research could examine the implications of including the category to assess program 
performance. Given that explorers do not concentrate in any one CTE field, the program 
questions to address are what are the expected outcomes of explorers and how could 
these outcomes be assessed within the accountability system? 


* How to reconcile different results from studies about the benefits of secondary CTE on 
academic coursetaking, achievement, and high school graduation? Some studies find no 
such benefits from CTE, but other studies find CTE strengthening academic coursetaking 
and graduation rates. Research could explore whether these differences arise from 
differences in study methodology, nature of CTE courses and programs, or differences in 
CTE population and context. Research could also move beyond only exploring average 
effects of CTE without taking into account program variation or implementation quality 
but instead focus on assessing the benefits from a set of defined and well-implemented 
programs. 


6.2 Final Report 


Perkins IV calls for an independent evaluation and assessment of career and technical education 
programs funded under the Act to examine both program implementation and outcomes. In 
response, the U.S. Department of Education, in consultation with the NACTE Independent 
Advisory Panel, formulated a research agenda that lays out the full range of studies and 
methodologies that are addressing the legislative requirements of the national assessment 
(Exhibit ES-1). The final report will present results from these studies in preparation for future 
discussions around the reauthorization of the Perkins Act. 
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The final report will include qualitative evidence of secondary and postsecondary CTE 
implementation and, when possible Tech Prep implementation, based on state and local surveys 
and case studies. Additionally, the final report will examine the secondary and postsecondary 
experiences of students who concentrated in CTE in high school using quantitative analysis, and 
earnings analysis when possible. 


In particular, the final report will (1) build on previous analyses of student participation in CTE 
by updating enrollment trends using the latest statistics from the National Center for Education 
Statistics (the National Assessment of Educational Progress, 2009 High School Transcript 
Study); (2) present descriptive statistics of secondary CTE teachers; (3) assess the effects of key 
legislative provisions on state and local implementation of Perkins IV particularly with respect to 
programs of study at the secondary and postsecondary levels; (4) examine the effectiveness of 
CTE participation in improving student outcomes; (5) establish how well new performance 
reporting requirements are working to promote accountability and program improvement at the 
secondary and postsecondary levels; and (6) assess the distribution and targeting of Perkins IV 
funding at both the secondary and postsecondary levels. 
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Appendix A. 


Perkins IV NACTE Requirements 
SECTION 114(d)(2)(B) 


CARL D. PERKINS CAREER AND TECHNICAL EDUCATION 
IMPROVEMENT ACT OF 2006 


NATIONAL ASSESSMENT OF CAREER AND TECHNICAL EDUCATION (NACTE) 
MANDATED BY THE CARL D. PERKINS CAREER AND TECHNICAL EDUCATION 
ACT OF 2006 (Perkins IV) (20 U.S.C. 2301 et seg. as amended by P.L. 109-270) 


Legislative requirements: 


Pursuant to section 114(d)(2)(B) of Perkins IV, “[t]he assessment required...shall include 
descriptions and evaluations of— 


“(i) the extent to which State, local, and tribal entities have developed, implemented, or 
improved State and local career and technical education programs assisted under the Act; 


“(ii) the preparation and qualifications of teachers and faculty of career and technical education 
(such as meeting State established teacher certification or licensing requirements), as well as 
shortages of such teachers and faculty; 


“(iii) academic and career and technical education achievement and employment outcomes of 
career and technical education, including analyses of— 


(1) the extent and success of the integration of rigorous and challenging academic and 
career and technical education for students participating in career and technical 
education programs, including a review of the effect of such integration on the 
academic and technical proficiency achievement of such students (including the 
number of such students receiving a secondary school diploma; and 


(II) the extent to which career and technical education programs prepare students, 
including special populations, for subsequent employment in high skill, high wage 
occupations (including those in which mathematics and science skills are critical), or 
for participation in postsecondary education; 


“(iv) employer involvement in, and satisfaction with, career and technical education programs 
and career and technical education students’ preparation for employment; 


“(v) the participation of students in career and technical education programs; 


“(vi) the use of educational technology and distance learning with respect to career and 
technical education and tech prep programs; and 
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“(vii) the effect of State and local adjusted levels of performance on the delivery of career and 
technical education services, including the percentage of career and technical education and 
tech prep students meeting the adjusted levels of performance described in section 113.” 
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The Department of Education’s mission is to promote student achievement and preparation for global 
competitiveness by fostering educational excellence and ensuring equal access. 
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